Search

In recent studies, deep learning methodologies have shown significant promise in crack detection. However, their practical implementation faces challenges due to the intricate diversity of structural surfaces and the inherent narrowness of cracks. To mitigate these problems, this paper introduces SegFormer, an efficient semantic segmentation model with hierarchical Transformer, for crack detection on concrete and asphalt surfaces in multiple scenarios. The combination of Cross-Entropy (CE) and Dice loss functions is employed to enhance the detection of fine cracks. Additionally, the paper presents an evaluation framework and discusses metrics for assessing crack segmentation results to provide a more precise and comprehensive analysis of model performance. Experimental results indicate that SegFormer outperforms Convolutional Neural Networks (CNNs) such as FCN, U-Net, and DeepLabV3 utilizing different backbones. Notably, the integration of multiple loss functions contributes to a more stable training process, expedites convergence, and yields enhanced results compared to models utilizing individual loss functions. • SegFormer is introduced for fine crack detection in various scenarios. • The combination of CE and Dice loss functions enhances crack detection. • A comprehensive model evaluation framework is presented. • Metrics for fine crack segmentation assessment are discussed. [ABSTRACT FROM AUTHOR]

Geological reports house a wealth of geological domain knowledge and expert experience knowledge, and the efficient extraction of geological entity relations from these texts holds great significance in constructing a geological knowledge graph. To address challenges such as overlapping entity relations, data imbalance, and the limited incorporation of contextual features in existing deep learning-based entity relation extraction methods, this paper proposes a relational graph convolutional network pointer model based on the pre-trained model of Robustly Optimize Bidirectional Encoder Representation from Transformers Pre-training Approach (RoBERTa). Specifically, our approach tackles the issue of overlapping entity relationships by identifying the primary entity and the subordinate entity associated with a specific relationship. Additionally, by incorporating a relationship graph convolutional network, we can capture multiple relationships among entities. Moreover, we leverage the Biaffine layer to accurately handle entity annotation boundaries, thereby enhancing the precision of entity classification. To address the issue of imbalanced data, we adopt the Focal Loss function during experiments. Furthermore, we construct a geological entity relation dataset comprising 24 types of relationships, which is generated by collecting seven regional geological reports and employing semi-automatic annotation techniques. Finally, the proposed algorithm was tested and comparison with several baseline models, the results demonstrate the superior performance of the proposed model in this paper. • In this paper, a geological entity relationship dataset is constructed. • In this paper, we propose a convolutional pointer network model for relational maps based on the pre-trained RoBERTa model and validate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals. [ABSTRACT FROM AUTHOR]

• We proposed a delineating field parcel model (TSANet) based on satellite image time series. • TSANet learn the relevance of spatial-spectral-temporal feature representation. • TSANet is robust across space and time. • TSANet performs better than pervious methods. • The time series data from the main growing period has a greater impact. Satellite image time series (SITS), such as Sentinel-2 imagery, plays a crucial role in the delineation of agricultural fields by reducing the impacts of ambiguities due to the spatial arrangement of field boundaries. Existing delineate field parcel models rely extensively on spatial features derived from single-date imagery. However, several studies have exploited the potential of SITS to effectively tackle the complexities associated with the intrinsic consistency between agricultural fields and their boundaries. This paper proposes a novel Two-Stream Attention convolutional neural network (TSANet) to capture the subtle difference between agricultural fields from SITS. Specifically, a field temporal semantic stream is introduced to adaptively leverage the significance of spatial-spectral-temporal feature representation associated with the location of agricultural parcels, especially where transitions in crop types take place. Considering the consistency between field parcels and their boundaries, we developed a field boundary prediction stream to enhance the extraction of edge features, particularly for the extraction of small and irregular agricultural parcels. Moreover, a field parcel refining block is employed to further enhance the geometric accuracy of agricultural fields. We conducted experiments on Sentinel-2 images from the Netherlands. Results showed that our approach produced a better layout of agricultural fields, with an average F1-score of 0.91 than the existing 3D-UNet, U-TEA, and BiConvLSTM. In addition, through the analysis of both quantitative and qualitative results, the stronger robustness of the model compared to other algorithms has been verified by temporal transfer and large-scale spatial prediction. We compared the difference between SITS and the corresponding composite images, which further verified the influence of temporal variation on the proposed approach. This paper provides a general guide for delineating agricultural parcels using SITS. [ABSTRACT FROM AUTHOR]

Soybean is a critical source of food, protein, and oil, and thus has received extensive research aimed at enhancing their yield, refining cultivation practices, and advancing soybean breeding techniques. Within this context, soybean pod counting plays an essential role in understanding and optimizing production. Despite recent advancements, the development of a robust pod-counting algorithm capable of performing effectively in real-field conditions remains a significant challenge. This paper presents a pioneering work of accurate soybean pod counting utilizing unmanned aerial vehicle (UAV) images captured from actual soybean fields in Michigan, USA. Specifically, this paper presents SoybeanNet, a novel point-based counting network that harnesses powerful transformer backbones for simultaneous soybean pod counting and localization with high accuracy. In addition, a new dataset of UAV-acquired images for soybean pod counting was created and open-sourced, consisting of 113 drone images with more than 260k manually annotated soybean pods. The images are taken from an altitude of approximately 13 ft, with angles between 53 and 58 degrees, under natural lighting conditions. Through comprehensive evaluations, SoybeanNet demonstrates superior performance over five state-of-the-art approaches when tested on the collected images. Remarkably, SoybeanNet achieves a counting accuracy of 84.51% when tested on the testing dataset, attesting to its efficacy in real-world scenarios. The publication also provides both the source code and the labeled soybean dataset, offering a valuable resource for future research endeavors in soybean pod counting and related fields. • Created and open-sourced the 1st UAV-acquired image dataset for soybean pod counting. • Pioneered the first study on precise soybean pod counting utilizing UAV images. • Demonstrated SoybeanNet's superiority over 5 state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Improving the accuracy of photovoltaic (PV) power forecasting is crucial to ensure more effective use of energy resources. Improvements are especially important for regions for which historical solar radiation data do not exist. This paper proposes a cyber-secure forecasting model called federated deep learning (FDL) to forecast PV power generation in various regions across Iran. The training process in each client is done by a convolutional neural network (CNN). Then, a generalizable global supermodel is generated based on the features extracted in each client, which has the ability to generalize and forecast for regions where there is no training data. Preserve data privacy and ideal performance against cyber-attacks are prominent features of the proposed method. The use of the proposed method is illustrated with a case study for Iran. The proposed FDL network is designed with 9 clients and three different scenarios were developed to test the robustness of the suggested method. In the first scenario, the PV power generation forecasting is done via the proposed technique and other conventional methods. The performance accuracy (R 2) of the generated global supermodel in this scenario for PV power generation forecasting in the regions of Khomein, Meybod, Varzaneh, Taleghan, and Shiraz are obtained as 0.981, 0.989, 0.986, 0.983, and 0.987 respectively. However, it was observed that other conventional deep learning-based models such as CNN and long short-term memory were not able to provide any forecasting for these regions. The second scenario models the scaling attack as a specific pattern of false data injection attack, to evaluate the performance of forecasting models against the data integrity attack. In the third scenario, cyber-attack detection is performed based on data visualization and image processing procedures. The results presented in different scenarios emphasize the high accuracy and generalizability of the global cyber-secure supermodel in PV power generation forecasting in different regions of Iran. [ABSTRACT FROM AUTHOR]

Domain shift is the problem in which trained models fail to maintain their performance when they confront new test domains. Cross-scene classification, a technique developed to overcome this challenge, has attracted significant research interest in the field of remote sensing (RS). By exploring the correlation between the source domain and the target domain, relevant models can have better generalization ability regarding the target domain. Nevertheless, Domain Adaptation (DA), as the main technique for cross-scene classification, necessarily requires access to the target samples to assist the model training, a condition that is difficult to satisfy in real-world applications. Domain Generalization (DG) has attracted increasing research attention in recent years. Given one or several source domain(s), DG tends to learn models that can perform well when dealing with unseen (inaccessible) target domains. DG can better deal with out-of-distribution generalization with fewer restrictions, making it a good fit for cross-scene classification. Notably, little research has been conducted on domain generalization in the field of remote sensing. Thus, we refer to this type of method as cross-scene generalization. Recent studies have shown that convolutional neural networks have a strong bias towards recognizing textures rather than shapes. Accordingly, we proposed a Style and Content Separation Network (SCSN) for RS image cross-scene generalization in this paper, which can improve the generalizability and discriminative capability. The Style and Content Separation (SCS) module uses instance normalization to obtain the content information and thereby ensure better generalization ability. Moreover, the residual feature, which contains the style information, can supplement the feature representations after refinement. We further proposed a separation loss to constrain the style and content separation process. Experimental results and relevant analysis demonstrate the effectiveness of the proposed SCSN on cross-scene generalization tasks. Code is available at https://github.com/WHUzhusihan96/SCSN. [ABSTRACT FROM AUTHOR]

Objective: Tissue mechanical properties are valuable markers for tissue characterization, aiding in the detection and staging of pathologies. Shear wave elastography (SWE) offers a quantitative assessment of tissue mechanical characteristics based on the SW propagation profile, which is derived from the SW particle motion. Improving the signal-to-noise ratio (SNR) of the SW particle motion would directly enhance the accuracy of the material property estimates such as elasticity or viscosity. Methods: In this paper, we present a 3-D multi-resolution convolutional neural network (MRCNN) to perform improved estimation of the SW particle velocity V z. Additionally, we propose a novel approach to generate training data from real acquisitions, providing high SNR ground truth target data, one-to-one paired to inputs that are corrupted with real-world noise and disturbances. Discussion: By testing the network on in vitro data acquired from a commercial breast elastography phantom, we show that the MRCNN outperforms Loupas' autocorrelation algorithm with an improved SNR of 4.47 dB for the V z signals, a two-fold decrease in the standard deviation of the downstream elasticity estimates, and a two-fold increase in the contrast-to-noise ratio of the elasticity maps. The generalizability of the network was further demonstrated with a set of ex vivo porcine liver data. Conclusion: The proposed MRCNN outperforms the standard autocorrelation method, in particular in low SNR regimes. [ABSTRACT FROM AUTHOR]

Recently, many efforts have been concentrated on land use land cover (LULC) classification due to rapid urbanization, environmental pollution, agriculture drought, frequent floods, and climate change. However, various aspects have attracted hyperspectral imaging due to there being informative discriminative features, such as spectral-spatial features. To this end, this paper is a comprehensive and systematic review of LULC classification using hyperspectral images by reviewing four significant research investigations. Moreover, the four investigations have addressed the following points: (1) the main components of the hyperspectral imaging, the modes of hyperspectral imaging with data acquisition methods, and the intrinsic differences between hyperspectral image and multispectral image, (2) the role of machine learning in LULC classification, and the standard deep learning methods: Convolution Neural Network (CNN), Stacked Autoencoder (SAE), Deep Belief Network (DBN), Recurrent Neural Network (RNN), and Generative Adversarial Network (GAN), (3) the standard benchmark hyperspectral datasets and the evaluation criteria, (4) the main challenges of LULC classification with the possible solutions for the limited training samples issue, the promising future directions, and finally the recent applications for LULC classification. [ABSTRACT FROM AUTHOR]

Recently, super-resolution (SR) techniques based on deep learning have taken more and more attention, aiming to improve the images and videos resolutions. Most of the SR methods are related to other fields of computer vision such as image classification, image segmentation, and object detection. Based on the success of the image SR task, many image SR surveys are introduced to summarize the recent work in the image SR domains. However, there is no survey to summarize the SR models for the lightweight image SR domain. In this paper, we present a comprehensive survey of the state-of-the-art lightweight SR models based on deep learning. The SR techniques are grouped into six major categories: include convolution, residual, dense, distillation, attention, and extremely lightweight based models. Also, we cover some other issues related to the SR task, such as benchmark datasets and metrics for performance evaluation. Finally, we discuss some future directions and open problems, that may help other community researchers in the future. [Display omitted] • Lightweight image super-resolution based on deep learning models are reviewed. • The network design, loss function, framework, and training datasets are considered. • The network design structure of the reviewed models are categorized into six types. • The future research directions of lightweight image super-resolution are indicated. [ABSTRACT FROM AUTHOR]

• Substituting the FFNs of the transformer encoders with locality modules enriches the local contexts in multi-object tracking. • The quantitative and qualitative research shows the strength of locality in transformer-based MOT systems. • The performance of pedestrian tracking is boosted when training the model with a mixture of detection and tracking datasets. Multi-object tracking (MOT) is a critical task in various domains, such as traffic analysis, surveillance, and autonomous vehicles. The joint-detection-and-tracking paradigm has been extensively researched, which is faster and more convenient for training and deploying over the classic tracking-by-detection paradigm while achieving state-of-the-art performance. This paper explores the possibilities of enhancing the MOT system by leveraging the prevailing convolutional neural network (CNN) and a novel vision transformer technique Locality. There are several deficiencies in the transformer adopted for computer vision tasks. While the transformers are good at modeling global information for a long embedding, the locality mechanism, which learns the local features, is missing. This could lead to negligence of small objects, which may cause security issues. We combine the TransTrack MOT system with the locality mechanism inspired by LocalViT and find that the locality-enhanced system outperforms the baseline TransTrack by 5.3% MOTA on the MOT17 dataset. [ABSTRACT FROM AUTHOR]

A tidal turbine can benefit from exquisitely designed morphing blades with a flexible trailing edge by mitigating up to 90% of the load fluctuation in harsh ocean environments, which reduces the overall cost of tidal energy. However, existing fluid–structure interaction (FSI) methods of resolving flow-induced deformation of the blades is computationally expensive, which poses an important challenge to effective morphing blade design. This paper presents a novel static FSI tool based on deep learning to cost-efficiently analyze the fluid–structure coupling of a hydrofoil. Specifically, adopting a convolutional neural network (CNN) to predict the fluid force and finite element method (FEM) to solve the solid structure response, a new CNN-FEM framework with an iterative scheme for solving the FSI problem is developed to achieve equilibrium between the fluid and structural forces. The new framework is used to predict the elastic deformation of the flexible blade section of the hydrofoil to demonstrate its effectiveness in the FSI evaluation. Comparison of the results to those produced by commercially developed software (i.e., Ansys Workbench) shows that this method yields extremely close prediction results of average equivalent stress and an accuracy of more than 92%. Moreover, it is 100 times more computationally efficient than the commercial Ansys Workbench software, requiring less than 3s for one-way FSI calculation. Taking advantage of this cost-effectiveness, the CNN-FEM can be used to achieve the accurate prediction of the deformation characteristics of the flexible hydrofoil under various flow scenarios that lay a foundation for advanced morphing blade design in the future. [ABSTRACT FROM AUTHOR]

The existing 3D deep learning methods adopt either individual point-based features or local-neighboring voxel-based features, and demonstrate great potential for processing 3D data. However, the point-based models are inefficient due to the unordered nature of point clouds and the voxel-based models suffer from large information loss. Motivated by the success of recent point-voxel representation, such as PVCNN and DRINet, we propose a new convolutional neural network, called Multi Point-Voxel Convolution (MPVConv), for deep learning on point clouds. Integrating both the advantages of voxel and point-based methods, MPVConv can effectively increase the neighboring collection between point-based features and also promote independence among voxel-based features. Extensive experiments on benchmark datasets such as ShapeNet Part, S3DIS and KITTI for various tasks show that MPVConv improves the accuracy of the backbone (PointNet) by up to 36% , and achieves higher accuracy than the voxel-based model with up to 34 × speedups. In addition, MPVConv outperforms the state-of-the-art point-based models with up to 8 × speedups. Also, our MPVConv only needs 65% of the GPU memory required by the latest point-voxel-based model (DRINet). The source code of our method is attached in https://github.com/NWUzhouwei/MPVConv. [Display omitted] • In this paper, we propose Multi Point-Voxel Convolution (MPVConv) which takes the advantages of both the voxel- and point-based methods, and can work with different backbones for a wide range of 3D tasks. • Our MPVConv increases not only the neighboring collection for point-based features, but also the independence among voxel-based features. • 1×1×1 3D CNN is adopted in fusion module to strengthen the information association and nonlinearity between point-based and voxel-based features. • We replace the tri-linear interpolation with nearest-neighbor interpolation to reduce the computation cost of voxel feature interpolation. [ABSTRACT FROM AUTHOR]

Forming a UAV cluster with multiple UAVs to collaborate in accomplishing tasks is the future development of battlefield. The UAV cluster can collaborate by communicating through inter-aircraft links and can quickly and accurately perform complex tasks such as path planning, collaborative reconnaissance, collaborative sensing and attack. In the process of Unmanned Aerial Vehicle (UAV) cooperative operation, obtaining the real-time attitude information of each UAV is preliminary to implement collaborative cooperation. However, in practice, due to the internal malfunction of sensors and mutual interference among UAVs, the acquired attitudes often suffer from data missing. This paper proposed a novel missing data reconstruction method for UAV's attitude in the case of sensor failures. The attitude data of UAV is obtained through an advanced UAV simulation platform AirSim. Through fusing the temporal convolutional network (TCN) and Bi-directional long short-term memory (Bi-LSTM), a deep regression framework is established for missing data construction. In addition, we evaluate the reliability of the proposed method by comparing with different baseline models and different combinations of data missing. The performance of our method is quantified using metrics of root-mean-square-error (RMSE), mean absolute error (MAE), determination coefficient (DC), mean absolute percentage error (MAPE), and symmetric mean absolute percentage error (SMAPE) score. The averaged performance values of our proposed method outperform other baseline algorithms. The results of the experiment show that the study could well-address the issue of missing data reconstruction and provide reliable attitude data. • A deep learning framework for UAV's missing data reconstruction is proposed. • TCN and Bi-LSTM is fused to reconstruct the missing data of UAV's attitude. • The performance for missing data reconstruction under different sensor failures is evaluated. [ABSTRACT FROM AUTHOR]

The goal of semantic segmentation is to segment the input image according to semantic information and predict the semantic category of each pixel from a given label set. With the gradual intellectualization of modern life, more and more applications need to infer relevant semantic information from images for subsequent processing, such as augmented reality, autonomous driving, video surveillance, etc. This paper reviews the state-of-the-art technologies of semantic segmentation based on deep learning. Because semantic segmentation requires a large number of pixel-level annotations, in order to reduce the fine-grained requirements of annotation and reduce the economic and time cost of manual annotation, this paper studies the works on weakly-supervised semantic segmentation. In order to enhance the generalization ability and robustness of the segmentation model, this paper investigates the works on domain adaptation in semantic segmentation. Many types of sensors are usually equipped in some practical applications, such as autonomous driving and medical image analysis. In order to mine the association between multi-modal data and improve the accuracy of the segmentation model, this paper investigates the works based on multi-modal data fusion semantic segmentation. The real-time performance of the model needs to be considered in practical application. This paper analyzes the key factors affecting the real-time performance of the segmentation model and investigates the works on real-time semantic segmentation. Finally, this paper summarizes the challenges and promising research directions of semantic segmentation tasks based on deep learning. [ABSTRACT FROM AUTHOR]

In this paper, we present an extensive review on deep learning methods for image restoration tasks. Deep learning techniques, led by convolutional neural networks, have received a great deal of attention in almost all areas of image processing, especially in image classification. However, image restoration is a fundamental and challenging topic and plays significant roles in image processing, understanding and representation. It typically addresses image deblurring, denoising, dehazing and super-resolution. There are substantial differences in the approaches and mechanisms in deep learning methods for image restoration. Discriminative learning based methods are able to deal with issues of learning a restoration mapping function effectively, while optimisation models based methods can further enhance the performance with certain learning constraints. In this paper, we offer a comparative study of deep learning techniques in image denoising, deblurring, dehazing, and super-resolution, and summarise the principles involved in these tasks from various supervised deep network architectures, residual or skip connection and receptive field to unsupervised autoencoder mechanisms. Image quality criteria are also reviewed and their roles in image restoration are assessed. Based on our analysis, we further present an efficient network for deblurring and a couple of multi-objective training functions for super-resolution restoration tasks. The proposed methods are compared extensively with the state-of-the-art methods with both quantitative and qualitative analyses. Finally, we point out potential challenges and directions for future research. [ABSTRACT FROM AUTHOR]

Reconstructing natural image from its corresponding compressive sensing (CS) measurements is an ill-posed problem. Learning accurate prior of desirable image is essential to solve this inverse problem in high quality, especially for CS reconstruction from noisy measurements. The existing learning-based methods cannot effectively simulate whole potential noise in the testing data during external learning, and cannot effectively exploit the information from internal testing data. In this paper, we present an effective convolutional neural network (CNN) based method for CS reconstruction from noisy measurements, which learns the deep prior from an external dataset and internal noisy testing data with Stein's unbiased risk estimator (SURE). Specifically, we first pre-train an arbitrary CNN for CS reconstruction with an external dataset to learn a common prior. Then, we utilize meta learning to find a generic initial parameter that is suitable for fast internal learning and adaption to various noisy measurements. Finally, we customize the learned network to learn a specific prior for each internal testing data under Gaussian noise or more general mixed Poisson-Gaussian noise. Experimental results show that the proposed method outperforms the state-of-the-art methods under both comprehensively quantitative metrics and perceptive quality. [ABSTRACT FROM AUTHOR]

Filter pruning has drawn extensive attention due to its advantage in reducing computational costs and memory requirements of deep convolutional neural networks. However, most existing methods only prune filters based on their intrinsic properties or spatial feature maps, ignoring the correlation between filters. In this paper, we suggest the correlation is valuable and consider it from a novel view: the frequency domain. Specifically, we first transfer features to the frequency domain by Discrete Cosine Transform (DCT). Then, for each feature map, we compute a uniqueness score, which measures its probability of being replaced by others. This way allows to prune the filters corresponding to the low-uniqueness maps without significant performance degradation. Compared to the methods focusing on intrinsic properties, our proposed method introduces a more comprehensive criterion to prune filters, further improving the network compactness while preserving good performance. In addition, our method is more robust against noise than the spatial ones since the critical clues for pruning are more concentrated after DCT. Experimental results demonstrate the superiority of our method. To be specific, our method outperforms the baseline ResNet-56 by 0.38 % on CIFAR-10 while reducing the floating-point operations (FLOPs) by 47.4 %. In addition, a consistent improvement can be observed when pruning the baseline ResNet-110: 0.23 % performance increase and up to 71 % FLOPs drop. Finally, on ImageNet, our method reduces the FLOPs of the baseline ResNet-50 by 48.7 % with only 0.32 % accuracy loss. [ABSTRACT FROM AUTHOR]

• We propose a new graph convolution network with subgraph embedding. • It extracts local information with subgraphs partitioned by Girvan Newman algorithm. • Experiments confirm the superiority of predicting mutagenicity in aromatic hydrocarbons. • The accuracy is verified with comparison of nine SOTA deep learning models. • The proposed method prevents about 15 %p of GCN's information dilution. An aromatic hydrocarbon refers to an organic material having a carbon ring such as benzene and a functional group in the carbon ring. As the industry develops, natural pollution becomes harsh, new compounds emerge, and the exposure to aromatic hydrocarbons is continuously increasing. Predicting mutagenicity is one of the crucial issues in reducing the risk because these organisms may have properties that penetrate the DNA of living things to cause mutations. Recently, the accuracy of mutation prediction has improved due to the power of deep learning. However, most conventional methods do not consider the characteristics of molecular aromatic hydrocarbons, which dilutes local information and results in a severe deterioration of the prediction performance. In this paper, we propose a method of exploiting subgraph convolution neural networks that enables the extraction of local information of a graph by partitioning it to maintain the detailed information. For extracting the features of molecules, we use the Girvan Newman algorithm to partition the graph according to the carbon ring and functional group and obtain the embedding vectors of the subgraphs as well as the original graph with graph convolution network (GCN). The embedding vectors are combined to represent the whole graph information and predict mutagenicity. Experiments with MUTAG, NCI1 and NCI109, datasets for predicting mutagenicity of molecules in graph structure, confirm that we successfully segment carbon rings and functional groups from molecular graphs and predict mutations using the partitioned graphs, leading to a 2 %p performance improvement. In addition, the proposed method has prevented about 15 %p of information dilution in GCN, and an analysis of the latent space of graphs reveals that the subgraphs extracted maintain the local information appropriately. [ABSTRACT FROM AUTHOR]

• Introduces the novel task of Sexual Facial Expression Recognition (SFER). • Triple-BigGAN; a conditional GAN with a generator, discriminator, and classifier. • SEA-Faces-30k; a new dataset with Erotic, Suggestive-Erotic, and Non-Erotic classes. • Evaluation using FER-2013, ExpW, MNIST, CIFAR-10, SVHN, and SEA-Faces-30k datasets. Automatic recognition of facial images showing erotic expressions can help to understand our social interaction and to detect non-appropriate images even when there is no nakedness present in them. This paper contemplates, for the first time, to exploit facial cues applied to automatic Sexual Facial Expression Recognition (SFER). With this goal, we introduce a new dataset named Sexual Expression and Activity Faces (SEA-Faces-30k) for SFER, which contains 30k manually labeled images under three categories: erotic, suggestive-erotic, and non-erotic. Deep Convolutional Neural Networks require large-scale annotated image datasets with diversity and variations to be properly trained. Unfortunately, gathering such a massive amount of data is not feasible in this area. Therefore, we present a new semi-supervised GAN framework named Triple-BigGAN, which learns a generative model and a classifier simultaneously. It learns both tasks in an end-to-end fashion while using unlabeled or partially labeled data. The Triple-BigGAN framework shows promising classification performance for the SFER task (i.e., 93.59 %) and other five benchmark datasets, i.e., FER-2013, CIFAR-10, Expression in-the-Wild (ExpW), Modified National Institute of Standards and Technology database (MNIST), and Street View House Numbers (SVHN). Next, we evaluated the quality of samples generated by Triple-BigGAN with a resolution of 256 × 256 pixels using Inception Score (IS) and Frechet Inception Distance (FID). Our approach obtained the best FID (i.e., 19.94 %) and IS (i.e., 97.98 %) scores on the SEA-Faces-30k dataset. Further, we empirically demonstrated that synthetic erotic face images generated by Triple-BigGAN could also help in improving the classification performance of deep supervised networks. [ABSTRACT FROM AUTHOR]

In recent years, deep learning (DL) methods have proven their efficiency for various computer vision (CV) tasks such as image classification, natural language processing, and object detection. However, training a DL model is expensive in terms of both complexities of the network structure and the amount of labeled data needed. In addition, the imbalance among available labeled data for different classes of interest may also adversely affect the model accuracy. This paper addresses these issues using a new convolutional neural network (CNN) based architecture. The proposed network incorporates both spatial and spectral information that combines two sub-networks: spatial-CNN and spectral-CNN. The spectral-CNN extracts spectral information, while spatial-CNN captures spatial information. Moreover, to make the features more robust, a multiscale spatial CNN architecture is introduced using different kernels. The final feature vector is formed by concatenating the outputs obtained from both spatial-CNN and spectral-CNN. To address the data imbalance problem, a generative adversarial network (GAN) was used to generate data for the underrepresented class. Finally, relatively a shallower network architecture was used to reduce the number of parameters in the network and improve the processing speed. The proposed model was trained and tested on Senitel-2 images for the classification of the debris-covered glacier. The results showed that the proposed method is well-suited for mapping and monitoring debris-covered glaciers at a large scale with high classification accuracy. In addition, we compared the proposed method with conventional machine learning approaches, support vector machine (SVM), random forest (RF) and multilayer perceptron (MLP). [ABSTRACT FROM AUTHOR]

When imaging at a long distance by ordinary visible cameras, the wavelength of visible light is easily interfered by fog or atmospheric effects, resulting in blurry or lost details in RGB images. However, NIR cameras are robust to the long distance imaging without color. In this paper, we propose long-range imaging using multispectral fusion of RGB and NIR images, called LRINet. We adopt unsupervised learning for the fusion to solve the absence of ground truth. LRINet is an end-to-end network based on convolutional neural networks (CNNs) that consists of three steps for the multispectral fusion: feature extraction, fusion and reconstruction. To align unpaired data and treat the discrepancy between RGB and NIR images, we construct WarpingNet to warp NIR features, and add it into the feature extraction. Since dark channel prior (DCP) provides distance from the camera by light transmission degree, we combine it with structure loss as the weight for fusion. To ensure the color fidelity, LRINet operates the fusion on the input of the RGB luma channel and NIR image. Experimental results show that LRINet produces natural-looking color images with clear details by successfully treating discrepancy between RGB and NIR images and outperforms state-of-art fusion models in term of both visual quality and quantitative measurements. • We propose long range imaging using multispectral fusion of RGB and NIR images, called LRINet. • LRINet consists of three steps for image fusion: feature extraction, fusion, and reconstruction. • We introduce WarpingNet into the feature extraction step to align the unpaired RGB and NIR images. • We combine DCP with structure loss to achieve long range imaging while keeping the original tone. • LRINet successfully treats discrepancy between two images and generates natural looking fusion images. [ABSTRACT FROM AUTHOR]

• A new approach for PD detection based on CNN deep features and i-vector features. • The impact of effective use of dynamic features for PD classification. • The significance of CNN deep features and i-vector features in terms of PD detection. Parkinson's disease (PD) corresponds to one of the most common neurological diseases in the world, which is mainly manifested by motor, cognitive, and language disorders. The change in the patient's voice is one of the most striking clinical signs and proves to be an element of interest to support the diagnosis and the assessment of PD. In this research paper, a new approach based on speech signal analysis is set forward to automatically detect Parkinson's disease. The approach evaluates two learning techniques, namely Support Vector Machines (SVM) and Convolutional Neural Networks (CNN), to classify data obtained from speech tasks. Two input data,i.e., the raw speech signal's values and the i-vector features of dimensions 100, 200 and 300 are extracted in this study. Eventually, an evaluation step is undertaken through the use of five evaluation metrics which are accuracy, precision, recall/sensitivity, specificity and f -score. The most pertinent obtained results for a test dataset composed of 28 participants are recorded as follows: an accuracy of 100%, precision of 0.99, recall/sensitivity of 0.98, specificity of 0.96 and f -score of 0.98. These findings corroborate that our approach can help in terms of PD detection. [ABSTRACT FROM AUTHOR]

• CNN architecture called MSFA-Net that acts as a texture feature extractor from raw images. • MSFA-Net requires to learn much fewer hyperparameters than state-of-the-art CNNs, and so reduces computation costs. • MSFA-Net outperforms several descriptors in multispectral texture classification and crop/weed recognition. • Texture features extracted from raw images are more discriminant than those extracted from demosaiced ones. Multispectral snapshot cameras fitted with a multispectral filter array (MSFA) acquire several spectral bands in one shot and provide a raw mosaic image in which a single channel value is available at each pixel. Texture features are classically extracted from fully-defined images that are estimated by demosaicing. Such an estimation may however cause spatio-spectral artifacts. Moreover, texture feature extraction becomes computationally inefficient and yields to high-dimensional features as the number of bands increases. In this paper, we propose an original approach based on a convolutional neural network called MSFA-Net to capture spatio-spectral interactions in raw images at reduced computation costs. Experiments of multispectral image classification and outdoor image segmentation show that the proposed approach outperforms several hand-crafted and deep learning-based feature extractors. [ABSTRACT FROM AUTHOR]

Infrared and visible image fusion aims to integrate complementary information from different types of images into one image. The existing image fusion methods are primarily based on convolutional neural network (CNN), which ignores long-range dependencies of images, resulting in the fusion network unable to generate images with good complementarity. Inspired by the importance of global information, we introduced the transformer technique into the CNN-based fusion network as a way to improve the entire image-level perception in complex fusion scenarios. In this paper, we propose an end-to-end image fusion framework based on transformer and hybrid feature extractor, which enables the network to focus on both global and local information, using the characteristics of transformer to compensate for the shortcomings of CNN itself. In our network, the dual-branch CNN module is used to extract the shallow features of images, and then the vision transformer module is used to obtain the global channel and spatial relationship in the features. Finally, the fusion results are obtained through the image reconstruction module. We calculate the loss in the features of different depths according to the different kinds of original images by using the pre-trained VGG19 network. The experimental results show the effectiveness of adding the vision transformer module. Compared with other traditional and deep learning methods, our method achieves state-of-the-art qualitative and quantitative experiments performance. [ABSTRACT FROM AUTHOR]

Analysis of point clouds through deep convolutional neural networks is an active area of research due to their massive real-world applications including autonomous driving, indoor navigation, robotics, virtual/augmented reality, unmanned aerial vehicles, and drone technology. However, capturing the fine-grained geometric and semantic properties for the underlying recognition task with raw unstructured point cloud is highly challenging due to the lack of explicit neighborhood relationship and sparsity among the points. In this paper, we have introduced a deep, hierarchical 3D point based architecture for object classification and part segmentation that is able to learn robust geometric features which remain invariant to both the geometry and orientation of the local patches. The proposed architecture consists of multiple layers of sampling, concentric annular convolution, pooling, and residual feature propagation blocks. In the skip connections of our deep residual design, we propose to use a combination of linear projection shortcut and nonlinear ReLU group normalization shortcut with batch normalization, to improve both the optimization landscape and the representational power. Our network achieves on par or even better than state-of-the-art results on synthetic and real-world object classification (i.e., ModelNet40 and ScanObjectNN) and part segmentation (i.e., ShapeNet-part) benchmark datasets. The implementation and results have been made available at ...https://github.com/Rabbia-Hassan/Deep_Annular_Residual_Feature_Learning_for_3dPointCloudsGitHub-link [ABSTRACT FROM AUTHOR]

The paper proposes a new class of nonlinear operators and a dual learning paradigm where optimization jointly concerns both linear convolutional weights and the parameters of these nonlinear operators. The nonlinear class proposed to perform a rich functional representation is composed by functions called rectified parametric sigmoid units. This class is constructed to benefit from the advantages of both sigmoid and rectified linear unit functions, while rejecting their respective drawbacks. Moreover, the analytic form of this new neural class involves scale, shift and shape parameters to obtain a wide range of activation shapes, including the standard rectified linear unit as a limit case. Parameters of this neural transfer class are considered as learnable for the sake of discovering the complex shapes that can contribute to solving machine learning issues. Performance achieved by the joint learning of convolutional and rectified parametric sigmoid learnable parameters are shown to be outstanding in both shallow and deep learning frameworks. This class opens new prospects with respect to machine learning in the sense that main learnable parameters are attached not only to linear transformations, but also to a wide range of nonlinear operators. [ABSTRACT FROM AUTHOR]

Segmentation of retinal vessel images is critical to the diagnosis of retinopathy. Recently, convolutional neural networks have shown significant ability to extract the blood vessel structure. However, it remains challenging to refined segmentation for the capillaries and the edges of retinal vessels due to thickness inconsistencies and blurry boundaries. In this paper, we propose a novel deep neural network for retinal vessel segmentation based on shared decoder and pyramid-like loss (SPNet) to address the above problems. Specifically, we introduce a decoder-sharing mechanism to capture multi-scale semantic information, where feature maps at diverse scales are decoded through a sequence of weight-sharing decoder modules. Also, to strengthen characterization on the capillaries and the edges of blood vessels, we define a residual pyramid architecture which decomposes the spatial information in the decoding phase. A pyramid-like loss function is designed to compensate possible segmentation errors progressively. Experimental results on public benchmarks show that the proposed method outperforms the backbone network and most state-of-the-art methods, especially in the regions of the capillaries and the vessel contours. In addition, performances on cross-datasets verify that SPNet shows stronger generalization ability. [ABSTRACT FROM AUTHOR]

The state-of-the-art performance for several real-world problems is currently reached by deep and, in particular, convolutional neural networks (CNN). Such learning models exploit recent results in the field of deep learning, leading to highly performing, yet very large neural networks with typically millions to billions of parameters. As a result, such models are often redundant and excessively oversized, with a detrimental effect on the environment in terms of unnecessary energy consumption and a limitation to their deployment on low-resource devices. The necessity for compression techniques able to reduce the number of model parameters and their resource demand is thereby increasingly felt by the research community. In this paper we propose the first extensive comparison, to the best of our knowledge, of the main lossy and structure-preserving approaches to compress pre-trained CNNs, applicable in principle to any existing model. Our study is intended to provide a first and preliminary guidance to choose the most suitable compression technique when there is the need to reduce the occupancy of pre-trained models. Both convolutional and fully-connected layers are included in the analysis. Our experiments involved two pre-trained state-of-the-art CNNs (proposed to solve classification or regression problems) and five benchmarks, and gave rise to important insights about the applicability and performance of such techniques w.r.t. the type of layer to be compressed and the category of problem tackled. [ABSTRACT FROM AUTHOR]

• We propose ReNAP for few-shot classification. • The APL module of ReNAP can learn class prototypes adaptively. • Experimental results demonstrate the superior classification performance of ReNAP. Traditional deep learning-based image classification methods often fail to recognize a new class that does not exist in the training dataset, particularly when the new class only has a small number of samples. Such a challenging and new learning problem is referred to as few-shot learning. In few-shot learning, the relation network (RelationNet) is a powerful method. However, in RelationNet and its state-of-the-art variants, the prototype of each class is obtained by a simple summation or average over the labeled samples. These simple sample statistics cannot accurately capture the distinct characteristics of the diverse classes of real-world images. To address this problem, in this paper, we propose the Relation Network with Adaptive Prototypical Learning method (ReNAP), which can learn the class prototypes adaptively and provide more accurate representations of the classes. More specifically, ReNAP embeds an adaptive prototypical learning module constructed by a convolutional network into RelationNet. Our ReNAP achieves superior classification performances to RelationNet and other state-of-the-art methods on four widely used benchmark datasets, FC100, CUB-200-2011, Stanford-Cars, and Stanford-Dogs. [ABSTRACT FROM AUTHOR]

Condition monitoring, fault diagnosis, and scheduled maintenance of wind turbines (WTs) are becoming a necessity to maximize their economic benefits and reduce their downtime. One of the critical faults in WTs is inter-turn short-circuit faults (ITSCF) in their rotating machines. Being dependent on higher wind speeds, offshore wind farms are more efficient than onshore ones, yet at the cost of higher ITSCF, more maintenance requirements, and difficulty in accessibility. Thus, they require an efficient non-contact fault diagnosis technique to enable operators to inspect all WT components with minimal contact. Among existing fault detection technologies, Infrared thermography (IRT) is considered as a non-invasive and non-destructive anomaly detection technique based on capturing thermal images by IR cameras, making it suitable for offshore harsh environments. Meanwhile, deep learning (DL) fault diagnosis approaches have proven to be promising intelligent tools with minimum human labor. This paper proposes an efficient and robust ensemble DL-based diagnosis method for ITSCF detection in induction rotating machines, besides identifying fault locations and short-circuit severity. The proposed technique is quite convenient for early diagnosis of offshore WT generator faults since it depends on IRT technology. Compared to previous studies, the proposed approach outperforms its counterparts since it integrates eight DL models, rather than using a single model, thus merging all their structural benefits altogether. Moreover, a cascaded feature fusion and selection procedure is introduced. First, the deep features extracted from the DL models are fused, then the most influential features are selected using a hybrid feature selection scheme. Thus, the classification accuracy is enhanced to reach 100% and meanwhile, the input features' size to the classifier is reduced, decreasing classification complexity and training time. Finally, no clustering or segmentation phases are required in the proposed technique, resulting in a further decrease in the diagnosis time and computation burden. • A CAD is proposed for faults diagnosis in induction generator-based offshore windfarms. • It fuses 8 DL models instead of using an individual one, thus boosting the classification ability. • The CAD employs a few features to perform diagnosis through a cascaded procedure. • First, features are fused, then significant features are selected via hybrid feature selection. • No segmentation or clustering stages are required which adds to the system's simplicity. [ABSTRACT FROM AUTHOR]

• Extracting complex feature information of TIS using multiple encodings. • Capturing hierarchical relationships between features using capsule network. • Improvements are made to the capsule network to enhance performance. • The proposed model improves the prediction performance compared to other models. Genes are the basic units of protein synthesis in organisms, and accurately identifying the translation initiation site (TIS) of genes is crucial for understanding the regulation, transcription, and translation processes of genes. However, the existing models cannot adequately extract the feature information in TIS sequences, and they also inadequately capture the complex hierarchical relationships among features. Therefore, a novel predictor named CapsNet-TIS is proposed in this paper. CapsNet-TIS first fully extracts the TIS sequence information using four encoding methods, including One-hot encoding, physical structure property (PSP) encoding, nucleotide chemical property (NCP) encoding, and nucleotide density (ND) encoding. Next, multi-scale convolutional neural networks are used to perform feature fusion of the encoded features to enhance the comprehensiveness of the feature representation. Finally, the fused features are classified using capsule network as the main network of the classification model to capture the complex hierarchical relationships among the features. Moreover, we improve the capsule network by introducing residual block, channel attention, and BiLSTM to enhance the model's feature extraction and sequence data modeling capabilities. In this paper, the performance of CapsNet-TIS is evaluated using TIS datasets from four species: human, mouse, bovine, and fruit fly, and the effectiveness of each part is demonstrated by performing ablation experiments. By comparing the experimental results with models proposed by other researchers, the results demonstrate the superior performance of CapsNet-TIS. [ABSTRACT FROM AUTHOR]

Food safety is an important public health issue, and deep learning (DL) algorithms can provide powerful tools and methods for food safety and authenticity detection. Compared with chemometric algorithms and traditional machine learning algorithms, the performances of DL algorithms are improved in many aspects. By learning and analyzing a large amount of data, DL models can improve the efficiency and accuracy of food safety and authenticity detection, helping to ensure the public health and safety. This paper reviews some commonly used chemometric algorithms, traditional machine learning algorithms, and popular DL algorithms. Among them, special attentions are paid to convolutional neural network (CNN), fully convolutional network (FCN) and generative adversarial network (GAN). Moreover, the auxiliary effect of GAN on CNN is highlighted. Finally, this paper revisits recent applications of DL algorithms in the field of food safety and authenticity detection, and prospects the challenges and future directions of DL algorithms in this field. Although DL has made many achievements in the field of food safety and authenticity detection, there is still a great potential for development. For example, the data augmentation function of GAN can assist CNN to obtain more training samples, thus improving the recognition rate. In addition, multimodal neural network (MNN) or multimodal attention network (MAN) can be also used to achieve the fusion of data from different modalities to further improve the robustness and accuracy of DL algorithms. [Display omitted] • Some popular DL algorithms in food safety and authenticity detection are introduced. • The improvements of DL algorithms to chemometric and ML algorithms are summarized. • Recent applications of DL algorithms in food safety and authenticity detection are reviewed. • Future prospects of DL algorithms in food safety and authenticity detection are discussed. [ABSTRACT FROM AUTHOR]

Because the carbon load inside a diesel particulate filters (DPF) affects the DPF regeneration, and the carbon load recognition is significant for the particulate matter (PM) emission control. It is necessary to investigate an on-board DPF carbon load recognition method because the carbon load cannot be directly measured by sensors. Aiming to build a DPF carbon load prediction model adopting the deep learning method, this paper proposes a DPF carbon load identification model based on different experimental parameters using a layered one dimension convolutional neural network (1D-CNN) method. To improve data validity, this paper adopts two data-processing methods. The data pre-processing adopts data splicing method to complete the construction of the original sample set, and the data after-processing uses wavelet packet transform method to establish the feature sample sets. The model adopts the optimal feature dataset constructed by three input parameters, i.e., temperature difference, pressure difference, and exhaust mass flow, and has both high training accuracy and test accuracy above 90 %. The pressure difference is the most important influencing input parameter, and the three-parameter sample set (Δ T + Δ P + Q) has great recognition accuracy and good model stability with the high training accuracy and test accuracy as well as less iteration. • A heavy-duty diesel engine was tested under various conditions of carbon load inside DPF. • The data processing was done by two methods: data splicing and wavelet packet transform. • We proposed a DPF carbon load identification model based on a one-dimension CNN method. • The model adopts 3 parameters, and the 3-parameter sample set has optimal performance. • The 1D-CNN model has both high training accuracy and test accuracy above 90 %. [ABSTRACT FROM AUTHOR]

Groundwater level (GWL) prediction is important for ecological protection and resource utilization; it helps in formulating policies for artificial groundwater recharge, modifying the number of extraction wells, etc., and can support sustainable human development as well as inform water resource management decisions. However, climate change, anthropogenic impacts, and the complex coupling between surface water and groundwater increase the difficulty of predicting groundwater levels. The model proposed in this paper combines external data as well as multiple models. The method leverages long and short-term memory (LSTM) and convolutional neural network (CNN) models, combined with secondary modal decomposition and slime mould algorithm (SMA), together with an adaptive weight module (AWM). The study applies this method to predict GWL for three different hydrological conditions in China, specifically for the Jinan Baotu Spring, Heihu Spring, and Zhongtianshe watershed of Taihu Lake. A comparison of metrics such as mean absolute error and Nash efficiency coefficient for single and hybrid models shows that the model in this paper is more advantageous than the single model and other hybrid models. The interpretability of the model is enhanced by SHAP values that demonstrate the degree of contribution of the input variables. This paper uses SHAP analyses to identify the key drivers affecting groundwater levels. These factors must be detected in order to develop groundwater resource protection measures. [Display omitted] • Multivariate fusion data including hydrology and meteorology are used as model input. • A secondary modal decomposition module for historical groundwater level data was utilized. • The neural network hyperparameters are optimized using the slime mould algorithm. • Aggregate subsets of prediction with adaptive modules rather than linear summation. • The interpretable SHAP model measures the degree of influence of external variables. [ABSTRACT FROM AUTHOR]

The detection of underwater fish targets is critical for ecological monitoring and marine biodiversity research. However, underwater fish detection is typically constrained by problems including low image quality and variable underwater surroundings. On behalf of further improving the underwater fish detection accuracy in complex underwater environments, this paper proposes a dual-path (DP) Pyramid Vision Transformer (PVT) feature extraction network named DP-FishNet. The backbone network DP-PVT composed from the PVT Network is made up of two feature extraction paths. The first represents the Vision Transformer path, which extracts global features to enhance the distinction between the foreground and background of underwater images. The second is the convolutional neural network path, which enhances the accuracy of detecting small targets by extracting local features. Additionally, to more effectively utilize the feature information extracted by the network, this paper provides a promising solution to employ the content-aware reassembly of features (Carafe) in the feature pyramid network (FPN). The seesaw loss is utilized as a classification loss to address the problem of unbalanced samples caused by the gap in the number of fish populations. According to the experimental findings, the AP and AP50 of the DP-FishNet are 76.0% and 95.2%, respectively. In comparison to currently available advanced two-stage detection algorithms, the quantity of computation and parameters is reduced by approximately 40%. DP-FishNet strengthens the ability to extract global and local features from underwater images and enhances feature reuse. DP-FishNet can be utilized to detect fish targets in actual and complicated underwater habitats. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new deformable convolutional network with attention mechanism to deal with the Chinese character of various dynasties. These ancient Chinese characters are hieroglyph with special spatial structure and gradually evolving into modern Chinese characters. However, because of the intrinsic limitations of traditional convolutional networks in the model, the geometric structure of the convolutional module is fixed. Therefore, we build a new architecture called attention-based deformable networks for this task. Besides, we construct a new dataset called the Chinese characters from Various Dynasties Dataset (CCDD), which includes the evolution of Chinese characters between major dynasties. Deformable convolution based on attention mechanism can obtain those more important offsets, resulting in better performance than the original deformable convolution. In the experimental stage, extensive experiments validate the performance of our approach. Firstly, the effectiveness of this network was verified on two text datasets, Mnist(99.38%) and notMnist(98.22%) both comparing with several modules. Finally, the best performance was achieved on the CCDD dataset(91.50%) compared to other classical deep learning networks. This is the first paper to apply deep learning methods to recognize and classify the characters of major dynasties in China. And archaeological workers need to be able to recognize Chinese characters from various periods in China, and our algorithm can assist them in completing this task to a certain extent. It can also help cultural relic enthusiasts roughly determine the age of cultural relics. [ABSTRACT FROM AUTHOR]

Marine ecosystems and their fish habitats are becoming increasingly important due to their integral role in providing a valuable food source and conservation outcomes. Due to their remote and difficult to access nature, marine environments and fish habitats are often monitored using underwater cameras to record videos and images for understanding fish life and ecology, as well as for preserve the environment. There are currently many permanent underwater camera systems deployed at different places around the globe. In addition, there exists numerous studies that use temporary cameras to survey fish habitats. These cameras generate a massive volume of digital data, which cannot be efficiently analysed by current manual processing methods, which involve a human observer. Deep Learning (DL) is a cutting-edge Artificial Intelligence (AI) technology that has demonstrated unprecedented performance in analysing visual data. Despite its application to a myriad of domains, its use in underwater fish habitat monitoring remains under explored. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring. In addition, we provide a comprehensive survey of key deep learning techniques for fish habitat monitoring including classification, counting, localisation, and segmentation. Furthermore, we survey publicly available underwater fish datasets, and compare various DL techniques in the underwater fish monitoring domains. We also discuss some challenges and opportunities in the emerging field of deep learning for fish habitat processing. This paper is written to serve as a tutorial for marine scientists who would like to grasp a high-level understanding of DL, develop it for their applications by following our step-by-step tutorial, and see how it is evolving to facilitate their research efforts. At the same time, it is suitable for computer scientists who would like to survey state-of-the-art DL-based methodologies for fish habitat monitoring. • Marine environments and fish habitats are widely surveyed using underwater cameras. • Deep learning (DL) offers an effective solution for processing underwater videos. • We write a tutorial on the key concepts and steps of DL for underwater monitoring. • We provide a survey of key DL-based visual fish monitoring methods/datasets. • We shed light on challenges/opportunities of DL for underwater fish monitoring. [ABSTRACT FROM AUTHOR]

This paper considers the multi-label thoracic abnormality classification with chest X-ray images. In clinical settings, Chest X-ray imaging is a general diagnostic tool applied to visualize numerous thoracic pathological changes. While deep learning techniques have been extensively tested in this field, certain challenges persist. The data in existing thoracic abnormality datasets is insufficient, and some diseases are extremely imbalanced. Meanwhile, the dependencies between different labels are often ignored. To tackle these issues head-on, this paper introduces two crucial modules: the group-wise spatial attention (GWSA) module and the label co-occurrence dependency (LCD) module, integrated with DenseNet121 backbone. Specifically, GWSA enhances the spatial features within distinct groups while keeping the between-group feature discrimination. LCD models the correlations between different thoracic abnormalities to refine the predicted probabilities. In conjunction with the DenseNet121 backbone, these two modules reach an average AUC score of 0.818 on Chest X-ray14 dataset, achieving state-of-the-art. Source code is available at https://github.com/YujiaKCL/Chest-Xray14-GWSA-LCD. [ABSTRACT FROM AUTHOR]

The fault management process is facilitated by equipping power distribution systems with automated devices, especially remote-controlled switches (RCSs). Although RCS plays a key role in improving the system's reliability, it imposes significant investment, installation, and maintenance costs. Hence, RCSs should be optimally located in distribution feeders for the highest profit. In previous works, reliability-oriented mathematical optimization models have been formulated to reach this goal. However, the number of test solutions exponentially grows with the problem size to find the globally optimal solution. This paper uses machine learning to propose a scalable and easy-to-implement model for optimal switch placement in real power distribution systems. At first, the features of candidate points for installing RCSs are introduced. Then, a learning model is applied to deeply explore the relationship between these features and optimal locations for installing RCSs. After training, the learning-based surrogate model directly determines the optimal RCS placement strategy in real power distribution systems by leveraging knowledge gained from past experiences. Simulation results demonstrate that the proposed surrogate model is approximately 29 times faster than the integer programming-based mathematical model, without a significant loss of accuracy, when implemented on a modified 11 kV network connected to Bus 4 of the Roy Billiton test system. This paper also employs explainable artificial intelligence (XAI) tools to select the most important features, where the Hamming loss is decreased by approximately 5%. [ABSTRACT FROM AUTHOR]

In this paper, machine learning (ML)/deep learning (DL) has been employed to predict the thermal runaway (TR) occurrence in Li-ion batteries (LIBs). State-of-the-art convolutional neural networks (CNNs) are adopted to forecast the evolution of the TR phenomenon with a classification approach of three distinct stages, namely, safe operation, critical condition of triggered the TR, and the actual TR occurrence. In addition, the "you look only once" (YOLO) object detection ML model is used to identify the location of the TR heat sources within the battery. The neural networks are trained on simulated thermal images, computed with the multiphysics modeling of LIBs. The multiphysics modeling approach comprises a coupled thermal, electrochemical (P2D model), and degradation sub-models. The degradation phenomenon leading to the TR considered in this paper is the solid electrolyte interface (SEI) formation/decomposition on the negative electrode. The proposed ML model exhibits high accuracy in predicting the TR evolution stages and heat source locations. The combined multiphysics and ML modeling approach developed in this work provides qualitative insights for 'on-the-fly' prediction of the TR, as well as a framework for extending data-driven methodologies to broad applications in electrochemistry. • Multiphysics modeling of thermal runaway (TR). • Simulated battery thermal imaging for training machine learning models for TR detection. • Multiclass classification of TR evolution: safe, critical, and TR stages. • Real-time detection of TR heat sources using object detection. [ABSTRACT FROM AUTHOR]

• This paper proposed a novel deep graph learning framework for airspace complexity prediction that can comprehensively capture the spatio-temporal dependencies between large-scale airspace sectors. • This paper constructed a multimodal adaptive graph convolution module that can effectively explore the multiple spatial dependencies and adaptively adjust the impact of different spatial modes on airspace complexity in a data-driven learning manner. • This paper designed a temporal convolution module based on a multiple-time-step self-attention mechanism to extract the mixed short- and long-term temporal patterns from both local and global perspectives. • This paper constructed a comprehensive dataset from real-world air traffic data within Chinese domestic airspace, ensuring its reliability through rigorous calibration with experienced air traffic controllers. Airspace complexity is defined as an essential indicator to comprehensively measure the safety of air traffic operational situations. A reliable prediction of airspace complexity can provide practical guidance for formulating air traffic management strategies and resource allocation. Although extensive efforts have been devoted to computing airspace complexity, previous studies can rarely model the multi-dimensional and combined spatio-temporal features within airspace complexity data. In this paper, we propose a multimodal adaptive spatio-temporal graph neural network to simultaneously explore the spatio-temporal dependencies in the airspace sector network. Specifically, we design a multimodal adaptive graph convolution module to effectively learn the diverse spatial relationships and adaptively adjust the impact of different spatial modes on airspace complexity in a data-driven manner. To model dynamic long-short-term temporal patterns, we develop a dilated causal convolution layer with a multiple-time-step self-attention mechanism to accurately predict airspace complexity over a longer time horizon. Extensive experiments on real-world air traffic datasets show that the proposed approach can harness differing spatial modes in achieving higher generalization performance across different temporal patterns, outperforming state-of-the-art methods in all prediction time horizons. [ABSTRACT FROM AUTHOR]

The aging of transformer oil–paper insulation (OPI) contributes significantly to transformer failure and reduced service life. Therefore, reliable and precise evaluation of the aging status of transformer insulation condition is crucial in averting transformer failure. This article proposes a novel deep learning-based aging condition assessment technique of transformers by combining FDS (Frequency domain spectroscopy) and squeeze-and-excitation-enabled convolutional neural network (SE-CNN). An intrinsic product-based transformation using a Gramian angular field is employed to dimensionally expand the 1-D FDS signals acquired from OPI samples into 2D images. Each feature extraction channel in the SE-CNN-based aging diagnosis framework is assigned a specific weight, enabling the model to concentrate on the most relevant features while discarding the rest. Furthermore, the effect of moisture variation on the model is also considered under three different conditions to ensure the consequence of compensation for the effect of moisture on FDS. A comparison with traditional machine learning models has also been made to ensure the viability of the proposed methodology. Additionally, the model is interpreted by creating explanation weight plots of the feature coefficients using a local interpretable model-agnostic explanations (LIME) method in Python. The findings demonstrate that the novel aging diagnosis framework visually assesses the transformer's OPI system with the highest accuracy of 99.12% in the test set. Furthermore, it facilitates the automatic extraction of deep aging features, thus providing better generalization and convergence capability. [ABSTRACT FROM AUTHOR]

Obtaining the aerodynamic performance of the turbine blade by Computational Fluid Dynamics (CFD) methods is accurate. However, it consumes time and computational resources. This paper proposes an evaluation method based on Convolutional Neural Network (CNN) and Artificial Neural Network (ANN) to obtain the aerodynamic performance of the turbine blade accurately and quickly. Compared with the existing data-driven modeling methods, this method innovatively introduces the Residual Network (ResNet), employs a transfer learning strategy for network design, and realizes the automatic extraction of blade profile features and non-parametric input. In processing boundary conditions, the ANN is utilized to fuse the blade profile features with the boundary conditions to realize the mapping between blade profile and aerodynamic performance under different conditions. In addition, to minimize the prediction deviation caused by the severely uneven distribution of the data set, we combined ensemble learning with transfer learning and proposed a two-step prediction strategy. The numerical simulations results show that the ResNet-ANN model established in this paper has a prediction relative error of 5 % on turbine blade aerodynamic parameters under various working conditions. The error is reduced by more than 90 % under −40°-10° incidence angle of incoming flow compared with the empirical model. • The method for predicting turbine blade profile loss is non-parametric. • The scheme is effective on both design and off-design conditions. • A two-step strategy is proposed for precise prediction on unevenly distributed data. • The TSP strategy is based on transfer learning and ensemble learning. • The approach is of high accuracy with considerable generalization. [ABSTRACT FROM AUTHOR]

• This paper presents a memory augmented convolutional neural network. • This paper utilizes the augmenting SOM as memory for better network generalization. • This paper instantiates the distributed knowledge representation with SOM and CNN. The long short-term memory (LSTM) network underpins many achievements and breakthroughs especially in natural language processing fields. Essentially, it is endowed with certain memory capabilities to boost its performance. Currently, the volume and speed of big data generation are increasing exponentially, and such data require efficient models to acquire memory augmented knowledge. In this paper, we propose a memory augmented convolutional neural network (MACNN) with utilizing self-organizing maps (SOM) as the memory module. First, we depict the potential challenge about just applying solely a convolutional neural network (CNN) so as to highlight the advantage of augmenting SOM memory for better network generalization. Then, we dissert a corresponding network architecture incorporating memory to instantiate the distributed knowledge representation machanism, which tactically combines both SOM and CNN. Each component of the input vector is connected with a neuron in a two-dimensional lattice. Finally, we test the proposed network on various datasets and the experimental results reveal that MACNN can achieve competitive performance, especially for bioimages datasets. Meanwhile, we further illustrate the learned representations to interpret the SOM behavior and to comprehend the achieved results, which indicates that the proposed memory-incorporating model can exhibit the better performance. [ABSTRACT FROM AUTHOR]

MVI is a risk assessment factor related to hepatocellular carcinoma (HCC) recurrence after hepatectomy or liver transplantation. The goal of this paper is to study the preoperative diagnosis of microvascular invasion (MVI) by using a deep learning algorithm in non-contrast T2 weighted magnetic resonance imaging (MRI) images instead of pathological images. Herein, an ensemble learning algorithm named H-DARnet—based on the difference degree and attention mechanism, combined with radiomics, for MVI prediction—is proposed. Our hybrid network combines the fine-grained, high-level semantic, and radiomics features and exhibits a rich multilevel-feature architecture composed of global-local-prior knowledge with suitable complementarity. The total loss function comprises two regularization items––the triplet and the cross-entropy loss function––which are selected for the triplet network and SE-DenseNet, respectively. The hard triplet sample selection strategy for a triplet network and data augmentation for small-scale liver image datasets in convolutional neural network (CNN) training is indispensable. For 200 patch level test samples (135 positive samples and 65 negative samples), our method can obtain the best prediction results, the AUC, sensitivity, and specificity were 0.826, 79.5% and 73.8%, respectively. The experiment results show that MVI can be predicted by using MRI images, and the proposed method is better than other deep learning algorithms and hand-crafted feature algorithms. The proposed ensemble learning algorithm is proved to be an effective method for MVI prediction. • The goal of this paper is to study the preoperative diagnosis of HCC MVI by using a deep learning algorithm on T2WI. • An algorithm named H-DARnet based on the difference degree and attention mechanism, combined with radiomics is proposed. • Our hybrid network exhibits a rich multilevel-feature architecture composed of global-local-prior knowledge. • Hard triplet sample selection strategy and data augmentation for small-scale datasets in CNN training is indispensable. • For 200 patch level test samples, our method can obtain the AUC, sensitivity, and specificity 0.826, 79.5% and 73.8%. [ABSTRACT FROM AUTHOR]

Low-light image enhancement studies how to improve the quality of images captured under poor lighting conditions, which is of real-world importance. Currently, convolutional neural network (CNN)-based methods with state-of-the-art performance have become the mainstream of research. However, most CNN-based methods improve the performance of the algorithm by increasing the width and depth of the neural network, which requires large computing device resources. In this paper, we propose a knowledge distillation method for low light image enhancement. The proposed method uses a teacher-student framework in which the teacher network tries to transfer the rich knowledge to the student network. The student network learns the knowledge of image enhancement under the supervision of ground truth images and under the guidance of the teacher network simultaneously. Knowledge transfer between the teacher-student network is accomplished by distillation loss based on attention maps. We designed a gradient-guided low-light image enhancement network that can be divided into an enhancement branch and a gradient branch, where the enhancement branch is learned under the guidance of the gradient branch to better preserve structural information. The teacher and student networks use a similar structure, but they have different model sizes. The teacher network has more parameters and more powerful learning capabilities than the student network. With the help of knowledge distillation, our approach can improve the performance of the student network without increasing the computational burden during the testing phase. The qualitative and quantitative experimental results demonstrate the superiority of our method compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Fine-grained visual categorization is an important task in computer vision. Prior works on fine-grained visual categorization have paid much attention to addressing intra-class variation and inter-class similarity. However, they rarely study that task from the perspective of probability distribution. In this paper, we propose a novel refined probability distribution module based on deep convolutional neural network. Our module computes the probability of an image by fully utilizing the similarity information between images. Firstly, we use deep neural networks to obtain the initial probability distribution and extract features. Then, we build a network whose inputs are features for calculating image-to-image similarity scores. Finally, our module refines the initial probability distribution based on an effective batch random walk operation with similarity scores. Our module can be plugged into many deep convolutional neural networks. Experimental results show that our approach outperforms state-of-the-art methods on the CUB-200–2011, FGVC-Aircraft and Stanford Cars datasets respectively. [ABSTRACT FROM AUTHOR]