Search

This paper delivers an exhaustive analysis of the fusion of multi-sensor technologies, including traditional sensors such as cameras, Light Detection and Ranging(LiDAR), Radio Detection and Ranging(RADAR), and ultrasonic sensors, with Artificial Intelligence(AI) powered methodologies in obstacle detection for Autonomous Vehicles(AVs). With the growing momentum in AVs adoption, a heightened need exists for versatile and resilient obstacle detection systems. Our research delves into study of literatures, where proposed approaches assimilate data from this diverse sensor suite, integrated through Deep Learning(DL) techniques, to refine AV performance. Recent advancements and prevailing challenges within the domain are thoroughly examined, with particular focus on the integration of sensor fusion techniques, the facilitation of real-time processing via edge and fog computing, and the implementation of advanced artificial intelligence architectures, including Convolutional Neural Networks(CNNs), Recurrent Neural Networks(RNNs), and Generative Adversarial Networks(GANs), to enhance data interpretation efficacy. In conclusion, the paper underscores the critical contribution of multi-sensor arrays and deep learning in enhancing the safety and reliability of autonomous vehicles, offering significant perspectives for future research and technological progress. [ABSTRACT FROM AUTHOR]

In recent years, decomposition-based combination models have been widely used in wind power prediction. This type of method decomposes the highly volatile wind power into some relatively smooth subsequences, which reduces the difficulty of modeling. However, this might use information from future data in advance, creating the illusion of high prediction accuracy. Therefore, this paper proposes a wind power ultra-short-term prediction model based on fixed scale dual mode decomposition (FSDMD) and deep learning networks. First, the wind power series after fixed scale blocking is decomposed using ensemble empirical mode decomposition (EEMD), and use the improved variational mode decomposition (VMD) based on Spearman rank order correlation coefficient (SROCC) to decompose the obtained high-frequency components twice. Then, the appropriate mode components were selected by calculating the SROCC and experimental analysis, and combined with the convolutional neural network (CNN) and the bidirectional long short-term memory (BiLSTM) network to train the model. Finally, the historical data of wind turbines in a wind farm in Northwest China is used for example verification, and the comparison with other models in the two scenarios of sufficient and insufficient features. The results show that the proposed FSDMD–CNN–BiLSTM model has high prediction accuracy in both scenarios. Especially in the scenario of insufficient features, compared with CNN-BiLSTM model, RMSE, MAE and MAPE are reduced by 8.20,14.24 and 0.15, respectively. In addition, this paper verifies that mode decomposition can improve the performance of prediction model without using future features, which provides ideas for solving similar problems. [ABSTRACT FROM AUTHOR]

Accurate detection of pavement markings at the pixel level is crucial for enhancing traffic safety. The majority of current advanced deep-learning networks predominantly focus on localized features, neglecting the global context of pavement image. Such networks often result in discontinuous segmentation outcomes and suboptimal recovery of local details. In this paper, a robust model named C-Transformer is proposed to provide an effective solution to this challenge. The contributions of this paper primarily involve two aspects. Firstly, the proposed C-Transformer is designed to succinctly integrate convolution operations and self-attention, facilitating a comprehensive understanding of essential features. Secondly, an efficient Feed-Forward Network called Inverse Residual Feed-Forward Network is also proposed in this paper and deployed in C-Transformer to improve latent representations. Experimental results demonstrate that, compared to other state-of-the-art networks, the proposed C-Transformer achieves a performance enhancement of 0.93% in F-measure and a 1.64% improvement in Intersection-Over-Union. In particular, the robustness and effectiveness of the C-Transformer in accurate pavement marking detection are proved through field test results. This paper illustrates the feasibility of employing a hybrid Convolutional neural network-Transformer-based network for automatic robust pavement marking detection under noisy conditions. [ABSTRACT FROM AUTHOR]

Gathering pipelines are one of the key upstream infrastructures in the gas industry that link production well to the processing plant. Leak detection is critical for ensuring the safety of pipeline transmission. The detection of small leakage in gathering pipelines consistently poses a formidable challenge. In this paper, a process model is built based on health data of supervisory control and data acquisition system from the actual operating pipeline. In the model structure, the convolutional neural network is used to extract the spatial features, the bi-directional long short-term memory is used to extract the temporal features, and the attention mechanism is employed to allocate the model's attention resources reasonably. Next, the residual between the entity pipeline's output data and the process model's output data is used as a monitoring indicator of the operating state of the pipeline. A clustering-based boundary determination method is proposed to recognize the centroid of normal and small leak conditions, and pipeline leak detection is performed by the Euclidean distance between the monitoring indicator and the centroid. This paper explores the feasibility of fast modeling and leak detection with limited hardware. Field tests for the validation of the proposed methods were implemented in two in-service natural gas gathering pipeline. The experimental results demonstrate that the proposed method significantly enhances the detection performance of small-size leak. The leak detection rates of 94.06% and 92.16% evinces the potency of the proposed method applied in the leak detection of gathering pipelines across diverse real-world scenarios. [ABSTRACT FROM AUTHOR]

Over the past decades, deep learning techniques have attracted increased attention from various research and industrial domains aligned with the development of Industry Internet-of-Things(IIoT). Specifically, with the advantage of data-driven methods, industrial organizations are seeking novel proactive strategies supported by analytic models to guarantee the quality of their production by observing degradation or predicting failure ahead of the occurrence of the component or asset. Predictive strategies are expected to promise the influence of unnecessary maintenance interruptions and mitigate the consequence of that, hence, extending the remaining useful life of products. This paper conducts a survey of the utilization of deep learning technologies on engineering applications where they provide satisfactory solutions with respect to specific data types or input signals. 106 primary papers are reviewed on deep learning–driven approaches which mainly explore five of the most popular architectures in the application of predictive maintenance. The main content of this paper summarizes the common advantages of each architecture and, accordingly, points out their limitations, as well as describes the application scopes of fully connected deep neural networks, convolutional neural networks, stacked autoencoders, deep belief networks, and deep recurrent neural networks. Based on the technique discussion for each of them, we intend to provide a comprehensive understanding and guidance of the appropriate usage of deep learning architectures to devise an effective predictive maintenance strategy for the scientific and industrial developers whose expertise lies in the prior domain knowledge of multi-source isomerization data. Moreover, the main content demonstrated the summarization of the decisive factor by which the incremental stages of the approaches were determined, fundamentally including the dataset specification, feature extraction, and the integration of deep learning approaches. [ABSTRACT FROM AUTHOR]

Gathering pipelines are one of the key upstream infrastructures in the gas industry that link production well to the processing plant. Leak detection is critical for ensuring the safety of pipeline transmission. The detection of small leakage in gathering pipelines consistently poses a formidable challenge. In this paper, a process model is built based on health data of supervisory control and data acquisition system from the actual operating pipeline. In the model structure, the convolutional neural network is used to extract the spatial features, the bi-directional long short-term memory is used to extract the temporal features, and the attention mechanism is employed to allocate the model's attention resources reasonably. Next, the residual between the entity pipeline's output data and the process model's output data is used as a monitoring indicator of the operating state of the pipeline. A clustering-based boundary determination method is proposed to recognize the centroid of normal and small leak conditions, and pipeline leak detection is performed by the Euclidean distance between the monitoring indicator and the centroid. This paper explores the feasibility of fast modeling and leak detection with limited hardware. Field tests for the validation of the proposed methods were implemented in two in-service natural gas gathering pipeline. The experimental results demonstrate that the proposed method significantly enhances the detection performance of small-size leak. The leak detection rates of 94.06% and 92.16% evinces the potency of the proposed method applied in the leak detection of gathering pipelines across diverse real-world scenarios. [ABSTRACT FROM AUTHOR]

The fault management process is facilitated by equipping power distribution systems with automated devices, especially remote-controlled switches (RCSs). Although RCS plays a key role in improving the system's reliability, it imposes significant investment, installation, and maintenance costs. Hence, RCSs should be optimally located in distribution feeders for the highest profit. In previous works, reliability-oriented mathematical optimization models have been formulated to reach this goal. However, the number of test solutions exponentially grows with the problem size to find the globally optimal solution. This paper uses machine learning to propose a scalable and easy-to-implement model for optimal switch placement in real power distribution systems. At first, the features of candidate points for installing RCSs are introduced. Then, a learning model is applied to deeply explore the relationship between these features and optimal locations for installing RCSs. After training, the learning-based surrogate model directly determines the optimal RCS placement strategy in real power distribution systems by leveraging knowledge gained from past experiences. Simulation results demonstrate that the proposed surrogate model is approximately 29 times faster than the integer programming-based mathematical model, without a significant loss of accuracy, when implemented on a modified 11 kV network connected to Bus 4 of the Roy Billiton test system. This paper also employs explainable artificial intelligence (XAI) tools to select the most important features, where the Hamming loss is decreased by approximately 5%. [ABSTRACT FROM AUTHOR]

The aging of transformer oil–paper insulation (OPI) contributes significantly to transformer failure and reduced service life. Therefore, reliable and precise evaluation of the aging status of transformer insulation condition is crucial in averting transformer failure. This article proposes a novel deep learning-based aging condition assessment technique of transformers by combining FDS (Frequency domain spectroscopy) and squeeze-and-excitation-enabled convolutional neural network (SE-CNN). An intrinsic product-based transformation using a Gramian angular field is employed to dimensionally expand the 1-D FDS signals acquired from OPI samples into 2D images. Each feature extraction channel in the SE-CNN-based aging diagnosis framework is assigned a specific weight, enabling the model to concentrate on the most relevant features while discarding the rest. Furthermore, the effect of moisture variation on the model is also considered under three different conditions to ensure the consequence of compensation for the effect of moisture on FDS. A comparison with traditional machine learning models has also been made to ensure the viability of the proposed methodology. Additionally, the model is interpreted by creating explanation weight plots of the feature coefficients using a local interpretable model-agnostic explanations (LIME) method in Python. The findings demonstrate that the novel aging diagnosis framework visually assesses the transformer's OPI system with the highest accuracy of 99.12% in the test set. Furthermore, it facilitates the automatic extraction of deep aging features, thus providing better generalization and convergence capability. [ABSTRACT FROM AUTHOR]

This paper proposes a real-time state of health (SOH) estimation model based on a deep learning (DL) framework. The proposed model is a combination of two different architectures; a one-dimensional convolutional neural network (1D-CNN) and a bidirectional gated recurrent unit (BiGRU). The hybrid CNN-BiGRU uses the 1D CNN layers to extract pertinent features from input data and then relies on the BiGRU layers for sequence learning in both directions. To account for all SOH indicators, the proposed approach uses the current, voltage, and temperature measurements, which are readily obtainable from the electric vehicle's battery management system (BMS). This prevents the complex and time-consuming feature extraction used in most related papers. Since the hyperparameters have a significant impact on the performance of neural network models, a Bayesian Optimization (BO) technique based on the Gaussian Process (GP) was considered to tune the CNN-BiGRU model hyperparameters. Accordingly, the objective function was able to converge to a low Mean Squared Error (MSE) of 1.2 × 10 − 5 in just 19 iterations. Afterward, to verify the accuracy of the optimized model, a Lithium-ion battery dataset with several discharge profiles provided by the National Aeronautics and Space Administration (NASA) was used. The obtained results demonstrated the accuracy and robustness of the proposed method compared to other commonly used models. The CNN-BiGRU model yielded a Mean Absolute Error (MAE) of 2.080% and a root-mean-square error (RMSE) of 2.516% in the case of the battery set #C, referring to a set with 70 cycles already used at 24 °C. Additionally, the End of life (EOL) indicator error of zero cycles for the same data. [ABSTRACT FROM AUTHOR]

Ensuring the safety and security of crowded places is a major concern for both the government and the public. Accurately and quickly estimating the number of people in a crowd is crucial for public safety, urban planning, and traffic monitoring. The existing methods have a minimum mean square error of 0.89 and require high storage space, making them inappropriate for crowd counting using low-computation and small-storage devices like single-board computers. Furthermore, these methods suffer from prediction time lag and are not suitable for live streaming. To tackle these challenges, this paper proposes a Deep Convolution Neural Network-based 'CrowdDCNN' model for crowd counting. This model reduces the value of mean square error by 0.29 and the size of the model by 80.01%. Additionally, the prediction time was decreased to 700 ms. Further, the 'NoLag' algorithm introduced in this paper is efficient in live crowd counting. Its O(1) time and space complexity make it appropriate for all devices, including single-board computers, laptops, and GPUs. The reported 0-p value during statistical analysis using Wilcoxon test, Friedman rank test, and paired t-test validates the superiority of the proposed model. [ABSTRACT FROM AUTHOR]

Structural topology optimization is a compute-intensive process due to several iterations of simulations required to evaluate the performance of the component during optimization. Deep learning (DL) based approaches can address this challenge, but these methods were demonstrated mainly using 2D shapes and, at best, in low-resolution 3D geometries (typically 3 2 3 ). Further, due to non-manufacturable geometric features, the predicted optimal geometries from DL may not be manufacturable, even using additive manufacturing. In this paper, we develop a DL framework using a multigrid convolutional neural network (CNN) to generate high-resolution topology-optimized 3D geometries with additional checks on the manufacturability of the predicted shapes. Our framework predicts the final optimal topology using the initial strain energy (objective function of structural topology optimization) and target volume fraction (material fraction to be preserved after optimization) as input. We train the network using a multigrid approach, which enables topology optimization at 12 8 3 resolution, which was previously computationally challenging. We first train the multigrid CNN at a lower resolution and then transfer the learned network to continue training at higher resolutions. We use a distributed deep learning framework on a GPU supercomputing cluster to further speed up the training time. Distributed DL significantly speeds up the training time by more than 4 × while achieving similar model performance. Finally, we check the optimal geometries for manufacturability using fused deposition modeling (FDM)-specific manufacturability constraints. The large training dataset (> 60,000 high-resolution topology optimization examples) will be released with the paper to enable further research on this topic. • Multigrid approach to train the neural network at fine resolution of 1283. • Demonstrate the manufacturability by 3D printing several sample models predicted our framework. • A data-parallel distributed deep learning framework to accelerate the training process. • A comprehensive high-resolution data set consisting of more than 60k optimal shapes. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of an effective and robust fusion representation method for foreground moving object detection. Many fusion representation learning approaches pay attention to the similarity measurement between the fusion results and source images in texture details and pixel intensity, ignoring the harmful information, e.g., noise, blur, and extreme illumination. Therefore, the aggregated features of infrared and visible images will introduce much harmful information, affecting the model performance of downstream visual tasks. This paper tackles these problems by proposing a contrastive fusion representation learning method for the foreground moving object detection task, which consists of two major modules: the upstream fusion representation module (FRM) and the downstream foreground moving object detection module (FODM). Unlike the traditional fusion optimization mechanism, the former aims to extract valuable features and reject harmful features via the maximum mutual information theory. The latter is a siamese convolutional neural network to detect foreground moving objects by aggregating the time-sequence images generated by FRM. Experimental results and comparisons with the state-of-the-art on three public datasets (i.e., TNO, MF, and cross-modal FOD dataset of infrared and visible images), validate the effectiveness, robustness, and overall superiority of the proposed contrast fusion representation learning method. Concisely, our contrastive fusion representation learning has gained 53. 9 % , 43. 2 % , 46. 4 % , 52. 3 % , 2. 2 % , 87. 1 % , 3. 5 % on EI, SF, DF, AG, MI, and Nabf metrics compared with the best competitors. [ABSTRACT FROM AUTHOR]

This paper presents a novel deep learning approach for reconstruction of local through-the-thickness fiber orientation distribution (FOD) in discontinuous long-fiber, prepreg-platelet molded composites (PPMC). The thermal-residual strains on composite surfaces are used as inputs to train a fully convolutional neural network, named U-Net, to predict spatially varying, local FOD. High fidelity synthetic data was generated via computational simulation of PPMC with stochastic material orientation state and was used for training the deep learning model. The proposed U-Net model allowed for rapid recognition of PPMC morphology by solving the inverse structural mechanics problem of determining the average fiber orientation through the composite thickness based on the provided surface strain measurements. Upon training and validation, the U-Net deep learning model was deployed to rapidly predict complex distributions of the local through-the-thickness FOD in PPMC. [ABSTRACT FROM AUTHOR]

The digitization of engineering drawings is crucial for efficient reuse, distribution, and archiving. Existing computer vision approaches for digitizing engineering drawings typically assume the input drawings have high quality. However, in reality, engineering drawings are often blurred and distorted due to improper scanning, storage, and transmission, which may jeopardize the effectiveness of existing approaches. This paper focuses on restoring and recognizing low-quality engineering drawings, where an end-to-end framework is proposed to improve the quality of the drawings and identify the graphical symbols on them. The framework uses K-means clustering to classify different engineering drawing patches into simple and complex texture patches based on their gray level co-occurrence matrix statistics. Computer vision operations and a modified Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) model are then used to improve the quality of the two types of patches, respectively. A modified Faster Region-based Convolutional Neural Network (Faster R-CNN) model is used to recognize the quality-enhanced graphical symbols. Additionally, a multi-stage task-driven collaborative learning strategy is proposed to train the modified ESRGAN and Faster R-CNN models to improve the resolution of engineering drawings in the direction that facilitates graphical symbol recognition, rather than human visual perception. A synthetic data generation method is also proposed to construct quality-degraded samples for training the framework. Experiments on real-world electrical diagrams show that the proposed framework achieves an accuracy of 98.98% and a recall of 99.33%, demonstrating its superiority over previous approaches. Moreover, the framework is integrated into a widely-used power system software application to showcase its practicality. The reference codes and data can be found at https://github.com/Lattle-y/AI-recognition-for-lq-ed.git Future work will focus on improving the generalizability of the proposed framework to different quality degradation scenarios and extrapolating the application to different engineering domains. [ABSTRACT FROM AUTHOR]

Object detection is one of the most important domains in computer vision tasks, which is an important branch of artificial intelligence. It aims at finding and locating the accurate position of objects in given pictures or videos. With the development of deep learning techniques, more powerful and robust algorithms have emerged to deal with multi-scale, high-level features to overcome the limitations of traditional pipeline of object detectors. The popularity of transformer framework enables larger capacity datasets by processing self-attention mechanism, and the object detection methods have evolved into a new era. This paper first reviews traditional object detection pipeline and brief history of deep learning, afterwards it focuses on the classification of deep learning-based object detection methods covering Convolution Neural Network based and transformer-based methods. Commonly used datasets and metrics are also covered in the next part. The Convolution Neural Network based methods mainly contain two-stage and one-stage detectors, Convolution Neural Network is the underlying structure of these methods convolutional stages are fundamental parts. Transformer-based models convert traditional object detection issues into end-to-end detection, which is widely used in dealing with images. Finally, the promising future of object detection areas are listed to show guidance on future work. [Display omitted] [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

The harmful utilization of DeepFake technology poses a significant threat to public welfare, precipitating a crisis in public opinion. Existing detection methodologies, predominantly relying on convolutional neural networks and deep learning paradigms, focus on achieving high in-domain recognition accuracy amidst many forgery techniques. However, overseeing the intricate interplay between textures and artifacts results in compromised performance across diverse forgery scenarios. This paper introduces a groundbreaking framework, denoted as Texture and Artifact Detector (TAD), to mitigate the challenge posed by the limited generalization ability stemming from the mutual neglect of textures and artifacts. Specifically, our approach delves into the similarities among disparate forged datasets, discerning synthetic content based on the consistency of textures and the presence of artifacts. Furthermore, we use a model ensemble learning strategy to judiciously aggregate texture disparities and artifact patterns inherent in various forgery types, thereby enabling the model's generalization ability. Our comprehensive experimental analysis, encompassing extensive intra-dataset and cross-dataset validations along with evaluations on both video sequences and individual frames, confirms the effectiveness of TAD. The results from four benchmark datasets highlight the significant impact of the synergistic consideration of texture and artifact information, leading to a marked improvement in detection capabilities. • We build an artifact and texture inconsistency detector for deepfake detection. • Texture coherence is analyzed via a novel mask extraction method and an autoencoder. • The use of an ensemble training strategy improved the ability to generalize. • Extensive experiments verified the effectiveness of the model we built. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a well-studied topic and one of the most challenging tasks in computer vision applications, such as autonomous driving. Deep learning approaches based on convolutional neural networks (CNN) have demonstrated exceptional success on this task in recent years. Despite this success, existing approaches are plagued by higher-order inconsistencies between the ground truth images and the ones predicted by the segmentation model. This paper proposes a novel post-processing scheme based on adversarial learning to counter these inconsistencies. Such a scheme can be combined with a variety of existing CNN-based semantic segmentation networks to improve their segmentation performances. The proposed scheme is a Two-Stream Conditional Generative Adversarial Network (TScGAN), with one stream having initial semantic segmentation masks predicted by an existing CNN, while the other stream utilizes scene images to retain high-level information under a supervised residual network structure. In addition, TScGAN incorporates a novel dynamic weighting mechanism, which leads to significant and consistent gains in segmentation performance. Several comparative tests on public benchmark driving databases, including Cityscapes, Mapillary, and Berkeley DeepDrive100K, demonstrate the effectiveness of the proposed method when used with state-of-the-art CNN-based semantic segmentation models. Furthermore, the ablation experiment proved the structural rationality of our two-stream structure. The code for TScGAN could be found at https://github.com/epan-utbm/TScGAN-for-Improving-Semantic-Predictions • Explores various post-processing techniques for enhancing semantic segmentation. • Novel Two-Stream Conditional GAN enhances urban drive scene segmentation. • Enhances feature extraction by implementing dynamic weighting within streams. • Evaluates the model against existing semantic segmentation methodologies. • Tests on Cityscapes, Mapillary, BDD100K reveal major IoU gains with proposed model. [ABSTRACT FROM AUTHOR]

The image noise generated by the welding process, such as arc light, splash, and smoke, brings significant challenges for the laser vision sensor-based welding robot to locate the weld seam and accurately conduct automatic welding. Currently, deep learning-based approaches surpass traditional methods in flexibility and robustness. However, their significant computational cost leads to a mismatch with the real-time requirement of automated welding. In this paper, we propose an efficient hybrid architecture of Convolutional Neural Network (CNN) and transformer, referred to as Dynamic Squeeze Network (DSNet), for real-time weld seam segmentation. More precisely, a lightweight segmentation framework is developed to fully leverage the advantages of the transformer structure without significantly increasing computational overhead. In this respect, an efficient encoder, which aims to increase its features diversity, has been designed and resulted in substantial improvement of encoding performance. Moreover, we propose a plug-and-play lightweight attention module that generates more effective attention weights by exploiting statistical information of weld seam data and introducing linear priors. Extensive experiments on weld seam images using NVIDIA GTX 1050Ti show that our approach reduces the number of parameters by 54x, decreases computational complexity by 34x, and improves inference speed by 33x compared to the baseline method TransUNet. DSNet achieves superior accuracy (78.01% IoU, 87.64% Dice) and speed performance (100 FPS) with lower model complexity and computational burden than most state-of-the-art methods. The code is available at https://github.com/hackerschen/DSNet. [ABSTRACT FROM AUTHOR]

In the face of uncontrolled offensive content on social media, automated detection emerges as a critical need. This paper tackles this challenge by proposing a novel approach for identifying offensive language in multilingual, code-mixed, and script-mixed settings. The study presents a novel multilingual hybrid dataset constructed by merging diverse monolingual and bilingual resources. Further, we systematically evaluate the impact of input representations (Word2Vec, Global Vectors for Word Representation (or GloVe), Bidirectional Encoder Representations from Transformers (or BERT), and uniform initialization) and deep learning models (Convolutional Neural Network (or CNN), Bidirectional Long Short Term Memory (or Bi-LSTM), Bi-LSTM-Attention, and fine-tuned BERT) on detection accuracy. Our comprehensive experiments on a dataset of 42,560 social media comments from five languages (English, Hindi, German, Tamil, and Malayalam) reveal the superiority of fine-tuned BERT. Notably, it achieves a macro average F 1 -score of 0.79 for monolingual tasks and an impressive 0.86 for code-mixed and script-mixed tasks. These findings significantly advance offensive language detection methodologies and shed light on the complex dynamics of multilingual social media, paving the way for more inclusive and safer online communities. [ABSTRACT FROM AUTHOR]

Accurate positioning of curtain wall frames is crucial for the automated installation of curtain wall modules. However, the current robot-based installation methods overly depend on visual guidance from operators, resulting in high costs and limiting construction efficiency. The development of deep learning has introduced an image segmentation approach that offers a new solution for the visual positioning of curtain wall frames. This paper proposes a context collaboration pyramid network to automatically segment curtain wall frames by incorporating context interaction and channel guided pyramid structure. The model adopts an "encoder-decoder" architecture with a feature interaction block strategically inserted between the encoder and decoder. Specifically, the encoder utilizes the pyramid pooling Transformer as a backbone to extract multi-level features from original RGB images. The decoder employs a channel guided pyramid convolution module to integrate multi-scale features and achieve finer prediction. Meanwhile, a context interaction fusion module between the features of adjacent levels was designed carefully to enhance the collaboration of the architecture. In addition, a benchmark dataset for the curtain wall frame segmentation task, consisting of 1547 images, was established. The dataset incorporates challenging scenarios, including strong lights, low contrast, and cluttered backgrounds. This method is evaluated on the collected dataset, and achieves an impressive accuracy of 97.30% and an F1-Score of 88.95%, outperforming other segmentation networks. Overall, the proposed method can extract target information accurately and efficiently and provide critical visual guidance for the robot, so as to promote the automatic installation level of the curtain wall module. [ABSTRACT FROM AUTHOR]

In the face of uncontrolled offensive content on social media, automated detection emerges as a critical need. This paper tackles this challenge by proposing a novel approach for identifying offensive language in multilingual, code-mixed, and script-mixed settings. The study presents a novel multilingual hybrid dataset constructed by merging diverse monolingual and bilingual resources. Further, we systematically evaluate the impact of input representations (Word2Vec, Global Vectors for Word Representation (or GloVe), Bidirectional Encoder Representations from Transformers (or BERT), and uniform initialization) and deep learning models (Convolutional Neural Network (or CNN), Bidirectional Long Short Term Memory (or Bi-LSTM), Bi-LSTM-Attention, and fine-tuned BERT) on detection accuracy. Our comprehensive experiments on a dataset of 42,560 social media comments from five languages (English, Hindi, German, Tamil, and Malayalam) reveal the superiority of fine-tuned BERT. Notably, it achieves a macro average F 1 -score of 0.79 for monolingual tasks and an impressive 0.86 for code-mixed and script-mixed tasks. These findings significantly advance offensive language detection methodologies and shed light on the complex dynamics of multilingual social media, paving the way for more inclusive and safer online communities. [ABSTRACT FROM AUTHOR]

Crude oil prices are susceptible to a variety of elements, including military, political, financial, and diplomatic forces, resulting in price fluctuations that are random, abrupt, and chaotic. Based on the chaotic nature of crude oil price series, this paper proposes a model for predicting crude oil prices that incorporates variational modal decomposition algorithm (VMD), phase space reconstruction technique (PSR), convolutional neural network (CNN), and bidirectional long and short-term memory network (BiLSTM). Specifically, the noises in the original data are removed using VMD. Next, the crude oil price is reconstructed using PSR. Then the reconstructed and denoised phase space is fed into the CNN-BiLSTM model for multi-step predictions. The empirical results show that the proposed model achieves the lowest MAPEs as 0.662%, 2.283%, 3.829%, and 4.44% and the lowest MSEs as 0.335, 3.942, 10.047 and 13.681. The Diebold Mariano (DM) test also demonstrates the proposed model's superiority in the domain of crude oil price forecasting. [ABSTRACT FROM AUTHOR]

Fault testing in the production line of automotive traction machines is essential to ensure the desired lifetime. Since repetitive partial discharges (PDs) caused by anomalies in the insulation system lead to premature breakdowns of electrical machines, a reliable PD detection is of great importance. This paper proposes deep learning (DL) methods to improve the discrimination of PD from background noise in comparison with the state-of-the-art amplitude based PD detection in the production line. First, a systematic data extraction and labeling procedure is introduced to obtain correctly labeled datasets from arbitrary PD measurements. In addition, datasets are enhanced with low signal-to-noise ratio PD pulses by applying a special data augmentation approach. 13 different convolutional, recurrent and fully connected neural networks are compared for various time-frequency representations of the input signals. Hyperparameters for input transform, network topology and solver are optimized for all 13 combinations to ensure a fair case study. As a result, the two-dimensional convolutional neural network with continuous wavelet transform achieves the best accuracy of around 99.76% on a test dataset of PD signals originating from previously not utilized test objects. All DL models considered in this comparison outperform the state-of-the-art threshold-based PD classification. Even for PD events with an amplitude close to the noise level, the detection rate is still around 95% for the best network. Furthermore, without applying the proposed data augmentation procedure, the DL models investigated are not able to distinguish small PD pulses from noise. • Deep learning to improve partial discharge classification of electrical machines. • Rise of partial discharge detection sensitivity and decrease in false alarm rate. • Data extraction, labeling and augmentation procedure to generate reliable datasets. • Comprehensive comparison of time-frequency transforms and network architectures. [ABSTRACT FROM AUTHOR]

This paper for the first time treats the interpretation of electrochemical noise time-frequency spectra as an image classification problem. It investigates the application of a convolutional neural network (CNN) for deep learning image classification of electrochemical noise time-frequency transient information. Representative slices of these spectra were selected by our transient analysis technique and served as input images for the CNN. Corrosion data from two types of pitting corrosion processes serve as test cases: AISI304 and AA2024-T3 immersed in a 0.01M HCl and 0.1M NaCl solution between 0 and 1ks after immersion, respectively. Continuous wavelet transform (CWT) spectra and modulus maxima (MM) are used to train the CNN, either individually or in a combined form. The classification accuracy of the CNN trained with the combined dataset is 0.97 and with the two individual datasets 0.72 (only CWT spectrum) and 0.84 (only MM). The ability to additionally classify a more progressed form of pitting corrosion of AA2024-T3 between 9 and 10ks after immersion indicates that the proposed method is sufficiently robust using combined datasets with CWT spectra and MM. The pitting processes can effectively be detected and classified by the proposed method. The most important contribution of the present work is to introduce a novel procedure that decreases the classical need for large amounts of raw data for training and validation purposes, while still achieving a satisfactory classification robustness. A relatively small number of individual signals thereby generates a multitude of input images that still contain all relevant kinetic information about the underlying chemo-physical process. [Display omitted] • Electrochemical noise interpretation is treated as an image classification problem. • Continuous wavelet transform spectra are divided into single transient spectra. • A small number of individual signals generates a multitude of input images. • Amount of raw data needed for training the convolutional neural network is reduced. • Modulus maxima increase the classification accuracy of pitting corrosion. [ABSTRACT FROM AUTHOR]

Fruit ripeness estimation models have for decades depended on spectral index features or colour-based features, such as mean, standard deviation, skewness, colour moments, and/or histograms for learning traits of fruit ripeness. Recently, few studies have explored the use of deep learning techniques to extract features from images of fruits with visible ripeness cues. However, the blackberry (Rubus fruticosus) fruit does not show obvious and reliable visible traits of ripeness when mature and therefore poses great difficulty to fruit pickers. The mature blackberry, to the human eye, is black before, during, and post-ripening. To address this problem, this paper proposes a novel multi-input convolutional neural network (CNN) ensemble classifier (MCE) for detecting subtle traits of ripeness in blackberry fruits. The multi-input CNN was created from a pre-trained visual geometry group 16-layer deep convolutional network (VGG16) model trained on the ImageNet dataset. The fully connected layers were optimized for learning traits of ripeness of mature blackberry fruits. The resulting model of about 600 K trainable parameters served as the base for building homogeneous ensemble learners t ensembled using the stack generalization ensemble (SGE) framework. The input to the network are images acquired with a stereo sensor using visible and near-infrared (Vis-NIR) spectral filters at wavelengths of 700 nm and 770 nm. Through experiments, the proposed model achieved 95.1 % accuracy on unseen sets and 90.2 % accuracy with in-field conditions. Further experiments reveal that machine sensory is highly and positively correlated to human sensory over blackberry fruit skin texture • Fruits that do not show obvious and reliable visible traits of ripeness when mature pose great difficulty in fruit ripeness estimation. • Hyperspectral imaging was investigated for evaluating blackberry fruit ripeness and determining the spectral wavelengths that optimally correspond to ripeness in mature blackberry fruit. • A novel multi-input convolutional neural network (CNN) ensemble classifier was proposed for detecting subtle traits of ripeness in images of blackberry fruits. • The proposed model with blackberry fruit images captured at spectral wavelengths of 700 nm and 770 nm achieved an 11.3 % and 16.2 % increase in test accuracies compared to the RGB-based model. • Bi-spectral imagery significantly improves the ripeness-sensing capabilities of deep learning techniques. • Machine sensory has a ripeness sensing capability that is highly and positively correlated to human sensory over blackberry fruit skin texture. [ABSTRACT FROM AUTHOR]

Advances in deep learning and convolutional neural networks (ConvNets) have driven remarkable face recognition (FR) progress recently. However, the black-box nature of modern ConvNet-based face recognition models makes it challenging to interpret their decision-making process, to understand the reasoning behind specific success and failure cases, or to predict their responses to unseen data characteristics. It is, therefore, critical to design mechanisms that explain the inner workings of contemporary FR models and offer insight into their behavior. To address this challenge, we present in this paper a novel template-inversion approach capable of reconstructing high-fidelity face images from the embeddings (templates, feature-space representations) produced by modern FR techniques. Our approach is based on a novel Deep Face Decoder (DFD) trained in a regression setting to visualize the information encoded in the embedding space with the goal of fostering explainability. We utilize the developed DFD model in comprehensive experiments on multiple unconstrained face datasets, namely Visual Geometry Group Face dataset 2 (VGGFace2), Labeled Faces in the Wild (LFW), and Celebrity Faces Attributes Dataset High Quality (CelebA-HQ). Our analysis focuses on the embedding spaces of two distinct face recognition models with backbones based on the Visual Geometry Group 16-layer model (VGG-16) and the 50-layer Residual Network (ResNet-50). The results reveal how information is encoded in the two considered models and how perturbations in image appearance due to rotations, translations, scaling, occlusion, or adversarial attacks, are propagated into the embedding space. Our study offers researchers a deeper comprehension of the underlying mechanisms of ConvNet-based FR models, ultimately promoting advancements in model design and explainability. [Display omitted] Above are example DFD reconstructions of selected sample images from a single subject in the LFW dataset. The rows display: original images (1st row), reconstructions from the VGG–VGG DFD model (2nd row), reconstructions from the VGG–ResNet DFD model (3rd row), reconstructions from the ResNet–VGG DFD model (4th row), and reconstructions from the ResNet–ResNet DFD model (5th row). • We present the Deep Face Decoder (DFD), a novel SOTA template inversion technique. • We use DFD to analyze the embedding space of ConvNet face recognition (FR) models. • Using DFD we explore the impact of image perturbations, occlusions, and attacks. • Through DFD we investigate various template construction strategies. [ABSTRACT FROM AUTHOR]

Implementing key engineering solutions to optimise the operation of energy industries requires daily electricity demand forecasting and including uncertainty, to promote markets insight analysis as part of their strategic planning, regulating and supplying electricity to consumers. This paper proposes hybrid artificial intelligence models combining convolutional neural networks (CNN) as a feature extraction algorithm with extreme learning machines (ELM) as a framework to predict electricity demand with confidence intervals generated by Kernel Density Estimation (KDE) approaches. In order to develop CELM-KDE model, time-lagged series of daily electricity demand with local climate variables based on the air temperature, atmospheric vapour pressure, evaporation, solar radiation, humidity and sea level pressures are used to train the proposed CELM-KDE hybrid model. In order to fully evaluate the newly developed model from a point-based, as well as a probabilistic prediction strategy, the observed and predicted electricity demand as well as the probability distribution of errors are analysed using KDE method that operates without prior data distribution assumptions. Based on observed and predicted electricity demand and the relevant probabilistic confidence intervals generated by the CELM-KDE model, the final results show that the proposed method attains significantly better probability interval predictions than traditionally-used point-based models. The proposed CELM-KDE model is demonstrated to be highly effective in providing a comprehensive coverage of predicted errors, as well as providing greater insights into the average bandwidth and detailed predicted electricity demand in the testing stage. The results also indicate that the proposed hybrid model is a reliable decision support tool to develop engineering solutions in area of energy modelling, monitoring and forecasting, which could potentially be useful to the industry policymakers. We show that the point-and probabilistic-based electricity demand predictive models can be employed as an effective tool to improve accuracy of forecasting and provision of insights for national electricity markets and key energy industry stakeholder application tools. [Display omitted] • A hybrid deep learning model for daily electricity demand prediction is proposed. • A Kernel Density Estimation is applied to derive probabilistic prediction intervals. • The algorithm quantifies uncertainties in point-based electricity demand predictions. • The proposed model outperforms deep learning models using statistical uncertainty indicators. [ABSTRACT FROM AUTHOR]

3D face reconstruction and dense alignment play a key role in digital humans. Learning-based approaches have been developed by employing cascaded convolutional neural networks for feature extraction and spatial-semantic relationship construction. However, these networks often struggle with accurately capturing the complex relationships among various facial components and attributes, particularly in unconstrained environments. Furthermore, the mappings modeled by convolutional weights are at low levels, which are usually implicit and local, and lack global discrimination. In this paper, we propose a global and instance-level perceived relationship matrices-based network (PRMNet) to recover high-fidelity 3D faces and perform accurate dense alignment in unconstrained environments. Specifically, a Key Information Extraction Module (KIEM) extracts crucial features from global and local feature banks. This reduces reasoning costs while maintaining high perceptual quality. Learnable global and instance-level perceived feature relationship matrices are then integrated into the Feature Relationship Reasoning Module (FRRM). This calibration combines key information from both the global macroscopic and sample-specific microscopic views, allowing accurate construction of spatial-semantic relationships to harvest both global discrimination and local relevance. Finally, we introduce a Spatial Reasoning and Guidance Module (SRGM), designed to recalibrate the joint weight responses of feature extraction and its enhancement paths using various attention mechanisms, thereby further enhancing global discrimination ability. Extensive quantitative and qualitative experiments on the benchmark datasets show that our PRMNet outperforms the state-of-the-art. Codes and all resources will be publicly available at https://github.com/Ray-tju/PRMNet. [ABSTRACT FROM AUTHOR]

This paper addresses the problem for fine-grained video super-resolution (FGVSR) to suppress temporal flickering caused by separately processed consecutive frames and enhance the quality of restored video frame details when upsizing videos. Some existing video SR methods fail to sufficiently utilize spatial-temporal information from input low-resolution (LR) videos, while others may generate undesirable artifacts or cannot well reconstruct image details. To overcome these problems, we present a novel deep learning framework for FGVSR, which takes a set of consecutive LR video frames and generate the corresponding super-resolved frames. Our deep FGVSR framework focuses on reconstructing missing information from the LR sources based on the proposed multi-frame alignment and refinement strategies. More specifically, we propose an alignment module, where multiple frames are aligned at feature level, to prevent the output videos from flickering. Then, we introduce a feature fusion module, where aligned features generated from our alignment module are fused and refined in a multi-scale manner. Finally, the proposed refinement module is used to reconstruct missing information based on the fused features. In addition, we also embed an image enhancement module on the skip connection from the input layer to the output layer of our network for further enhancing the SR results. Experimental results show that the proposed deep FGVSR, compared with existing deep learning-based VSR methods, achieves state-of-the-art performances on the three well-known benchmarks, including REDS, Vid4, and Vimeo90k. More specifically, compared with the state-of-the-art VSR methods in our experiments, our FGVSR achieves quantitative improvements from 0.70 dB to 9.54 dB in PSNR. On the other hand, our method has also been shown to be efficient to other image restoration tasks, such as image inpainting. • End-to-end trainable deep learning-based fine grained video super-resolution model is proposed. • The temporal dependency is implicitly obtained without performing explicit motion compensation. • Image details are reconstructed and further sharpened by our refinement and enhancement modules. • Adaptively integrated into other deep image restoration model for further boosting performances. [ABSTRACT FROM AUTHOR]

Multi-view stereo (MVS) reconstruction is a key task of image-based 3D reconstruction, and deep learning-based methods can achieve better results than traditional algorithms. However, most of the current deep learning-based MVS methods use convolutional neural networks (CNNs) to extract image features, which cannot achieve the aggregation of long-distance context information and capture robust global information. In addition, in the process of fusing depth maps into point clouds, the confidence filters will filter out the depth values with low confidence in weak texture areas. These problems will lead to the low completeness of 3D reconstruction of weak texture and texture-less areas. To address the above problems, this paper proposes SA-MVSNet based on the PatchmatchNet with a self-attentive mechanism. First, we design a coarse-to-fine network framework to advance depth map estimation. In the feature extraction network, a module with a pyramid structure based on Swin Transformer Block is used to replace the original Feature Pyramid Network (FPN), and the self-correlation between weak texture areas is enhanced by applying a global self-attention mechanism. Then, we also propose a self-attention-based adaptive propagation module (SA-AP), which applies a self-attention calculation within depth value propagation window to obtain the relative weight values of current pixel and others, and then adaptively samples the depth values of neighbors on the same surface for propagation. Experiments show that SA-MVSNet has significantly improved the completeness of 3D reconstruction for the images with weak texture on DTU (provided by Danish Technical University), BlendedMVS, and Tanks and Temple datasets. Our code is available at https://github.com/miaowang525/SA-MVSNet. • SA-MVSNet proposed a multi-scale feature extraction module based on Swin Transformer. • A global self-attention mechanism is applied to enhance the self-correlation between weak textured areas. • SA-MVSNet proposed a self-attention-based adaptive propagation module (SA-AP). • Apply window self-attention mechanism to enhance the confidence of depth value. • SA-MVSNet has significantly improved the completeness of 3D reconstruction on three datasets. [ABSTRACT FROM AUTHOR]

Deep learning methods have achieved a lot of success in various applications involving converting wearable sensor data to actionable health insights. A common application areas is activity recognition, where deep-learning methods still suffer from limitations such as sensitivity to signal quality, sensor characteristic variations, and variability between subjects. To mitigate these issues, robust features obtained by topological data analysis (TDA) have been suggested as a potential solution. However, there are two significant obstacles to using topological features in deep learning: (1) large computational load to extract topological features using TDA, and (2) different signal representations obtained from deep learning and TDA which makes fusion difficult. In this paper, to enable integration of the strengths of topological methods in deep-learning for time-series data, we propose to use two teacher networks — one trained on the raw time-series data, and another trained on persistence images generated by TDA methods. These two teachers are jointly used to distill a single student model, which utilizes only the raw time-series data at test-time. This approach addresses both issues. The use of KD with multiple teachers utilizes complementary information, and results in a compact model with strong supervisory features and an integrated richer representation. To assimilate desirable information from different modalities, we design new constraints, including orthogonality imposed on feature correlation maps for improving feature expressiveness and allowing the student to easily learn from the teacher. Also, we apply an annealing strategy in KD for fast saturation and better accommodation from different features, while the knowledge gap between the teachers and student is reduced. Finally, a robust student model is distilled, which can at test-time uses only the time-series data as an input, while implicitly preserving topological features. The experimental results demonstrate the effectiveness of the proposed method on wearable sensor data. The proposed method shows 71.74% in classification accuracy on GENEActiv with WRN16-1 (1D CNNs) student, which outperforms baselines and takes much less processing time (less than 17 sec) than teachers on 6k testing samples. [ABSTRACT FROM AUTHOR]

The aim of this paper is to investigate emotion recognition using a multimodal approach that exploits convolutional neural networks (CNNs) with multiple input. Multimodal approaches allow different modalities to cooperate in order to achieve generally better performances because different features are extracted from different pieces of information. In this work, the facial frames, the optical flow computed from consecutive facial frames, and the Mel Spectrograms (from the word melody) are extracted from videos and combined together in different ways to understand which modality combination works better. Several experiments are run on the models by first considering one modality at a time so that good accuracy results are found on each modality. Afterward, the models are concatenated to create a final model that allows multiple inputs. For the experiments the datasets used are BAUM-1 ((Bahçeşehir University Multimodal Affective Database - 1) and RAVDESS (Ryerson Audio–Visual Database of Emotional Speech and Song), which both collect two distinguished sets of videos based on the different intensity of the expression, that is acted/strong or spontaneous/normal, providing the representations of the following emotional states that will be taken into consideration: angry, disgust, fearful, happy and sad. The performances of the proposed models are shown through accuracy results and some confusion matrices, demonstrating better accuracy than the compared proposals in the literature. The best accuracy achieved on BAUM-1 dataset is about 95%, while on RAVDESS it is about 95.5%. [Display omitted] • Emotion recognition through multimodal architectures. • Comparison of early to late fusion of 1-input models. • Synchronization of video and audio channels. [ABSTRACT FROM AUTHOR]

Understanding thermal dynamics and obtaining the computational model of residential buildings enable its scaled application in energy retrofits, control optimization and decarbonization. In this paper, we present a deep learning approach to model building thermal dynamics with smart thermostat data collected from residential buildings, with the goal to investigate model generalizability. In the first stage, we developed and compared different Deep Learning architectures including Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) models and CNN-LSTM to predict indoor air temperature in a multi-step time horizon. In the second stage, we implemented a Transfer Learning (TL) process, which aims to improve the prediction performance on a new set of buildings (targets), exploiting the knowledge of related or similar buildings (sources). Different TL strategies and source model identification methods were investigated. The study showed that the CNN-LSTM performed the best among the architectures compared, with an average Mean Absolute Error (MAE) of 0.26 °C for one-hour-ahead (twelve 5-min future steps) predictions. Furthermore, the results showed that freezing the LSTM layer and fine-tuning the other layers of the CNN-LSTM achieved the best performance among four TL strategies, which further improved the performance with respect to a machine learning approach by 10%, and proving the effectiveness and generalizability of the proposed approach. A comparison of three different source model identification methods showed that randomly selecting source models constrained by similar building characteristics can provide good TL performance while retaining simplicity comparing with other quantitative source identification methods. • Investigated deep transfer learning techniques for residential thermal dynamics using smart thermostat data. • Compared diverse neural network architectures for indoor temperature prediction. • Explored transfer learning to enhance model accuracy. • Evaluated four transfer learning methods and three source model identification approaches. • Achieved high accuracy and generalizability with the proposed deep transfer learning approach. [ABSTRACT FROM AUTHOR]

Induction motors are widely used in various industrial applications due to their simplicity, robustness, and high efficiency. In recent years, deep learning approaches have shown great potential for detecting and diagnosing faults in induction motors. However, it was observed that the convolutional neural network (CNN) might miss the crucial long-range temporal dependencies of the significant features. At the same time, the recursive neural network (RNN) may face issues such as vanishing gradient and less computational efficiency if the data is too long with complex patterns. This paper proposes a novel deep-learning model that combines a one-dimensional CNN and an RNN into two independent pipelines. The spatial, temporal, long-term, and short-term features can be extracted from the raw input signal. A multi-head mechanism, a state-of-the-art deep learning technique in natural language processing (NLP), enables the model to focus more on relevant features and prevent overfitting. Specifically, our method simplifies the conventional fault diagnosis workflow while maintaining competitive accuracy compared to several state-of-the-art methods. By leveraging the ability to detect faults directly from raw, unprocessed electrical signals, our approach offers a practical, cost-effective solution for real-time fault detection in industrial applications. This innovation has the potential to significantly reduce downtime and the associated operational costs, thereby providing immediate practical value in various industrial settings. [ABSTRACT FROM AUTHOR]

Most pavement crack detection methods based on deep learning rely too much on pixel-wise labels, and are facing with sample imbalance problem. This paper proposes a three-stage pavement crack detection framework with positive sample augmentation by an autoencoder-deep convolutional generative adversarial network (E-DCGAN). Positive samples are the images labeled as crack in the training datasets. Firstly, crack regions are located using graph based on visual saliency (GBVS). Secondly, a convolutional neural network (CNN) model is designed to recognize crack pixels in the located crack regions. The E-DCGAN is explored to generate crack images, which are then added as augmented positive samples in the training dataset of the CNN model. The trained CNN model takes the located crack regions as inputs, outputs the recognized crack pixels. Finally, fine crack detection is realized through region growing. Crack contours are completely depicted pixel-wise based on the recognized crack pixels and the Breadth-first search. Experiments are carried out on two benchmark datasets and in practical applications. The results showed that the framework could detect cracks at the pixel level. Its performance is improved compared with standard fully convolutional network (FCN)-based and U-shaped CNN (U-Net)-based crack detection methods. In addition, the standard FCN-based and U-Net-based methods require accurate pixel-wise annotations, while the proposed framework only requires image-wise annotations. It can be concluded that the proposed framework not only effectively avoids the dependence on pixel-level annotations, improves the sample imbalance issues, but also achieves competitive detection performance over the FCN and U-Net based methods. • Proposed a crack image augmentation method based on a novel GAN model. • Designed a three-stage detection framework to locate cracks in pixel from coarse to fine. • Verified necessity of the augmentation and feasibility of the proposed framework. [ABSTRACT FROM AUTHOR]

Traditional machine learning methods require the assumption that training and test data are drawn from the same distribution, which proves challenging in real-world applications. Moreover, deep learning models require a substantial amount of labeled data for training in classification tasks and limited samples may lead to overfitting. In many real-world scenarios, there is an insufficient supply of labeled samples within the target domain for learning. Transfer learning offers an effective solution, allowing knowledge from a source domain to be transferred to a target domain. Additionally, data augmentation enhances model generalization by increasing data samples, particularly beneficial when dealing with limited target domain data. In this paper, we synergistically enhance the model's performance on classification tasks by integrating transfer learning techniques with a data augmentation strategy. By conducting numerous experiments across various datasets, we verified the effectiveness of our proposed approach. [ABSTRACT FROM AUTHOR]

This paper introduces a novel integrated microstructure design methodology that replaces the common existing design approaches for multifunctional composites: 1) reconstruction of microstructures, 2) analyzing and quantifying material properties, and 3) inverse design of materials. The problem of microstructure reconstruction is addressed using the diffusion-based generative model (DGM), which is a state-of-the-art generative model formulated with a Markovian diffusion process. Then, the conditional formulation of DGM is introduced for guidance to the embedded desired material properties with a transformer-based attention mechanism, which enables the inverse design of multifunctional composites. Furthermore, a convolutional neural network (CNN)-based surrogate model is utilized to facilitate the prediction of linear/nonlinear material properties for building microstructure-property linkages. Combined, the proposed artificial intelligence-based design framework enables large data processing and database construction that is often not affordable with resource-intensive finite element method (FEM)-based direct numerical simulation (DNS) or iterative reconstruction methods. It is worth noting that the proposed DGM-based methodology is not susceptible to unstable training or mode collapse, which are common issues in generative models that are often difficult to address even with extensive hyperparameter tuning. An example case is presented to demonstrate the effectiveness of the proposed approach, which is designing mechanoluminescence (ML) particulate composites. The results show that the designed ML microstructure samples with the proposed generative and surrogate models meet the multiple design requirements (e.g., volume fraction, elastic constant, and light sensitivity). This assessment demonstrates that the proposed integrated methodology provides an end-to-end solution for practical material design applications. [Display omitted] • An integrated microstructure design framework utilizing the diffusion-based generative model (DGM) is proposed. • The problem of microstructure reconstruction is addressed using the DGM with forward/reverse Markovian diffusion processes. • The conditional formulation of DGM is introduced to enable the inverse design of multifunctional composites. [ABSTRACT FROM AUTHOR]

Detecting malicious attacks in Industrial Internet of Things (IIoT) is crucial to minimize downtime and financial losses. However, existing deep learning (DL) research faces limitations in reliability and generalization due to reliance on a single dataset. This single dataset approach hinders the effectiveness of the models when applied to new and unseen datasets. This paper addresses the issue using multiple-domain learning, creating a generalized DL framework for IIoT traffic classification. It combines three datasets: Edge-IIoTSet, WUSTL-IIoT-2021, and X-IIoTID. The proposed framework has two stages. The first stage involves the fusion of different datasets into a common feature space. To achieve this, an autoencoder architecture is proposed to match the dimensionality of the datasets into a common feature space. Subsequently, a modified locally linear embedding is used as a manifold alignment to ensure statistical matching of multiple datasets such that the maximum mean discrepancy between multiple datasets is minimized to compare and combine datasets effectively. The second stage presents a hybrid DL model with a convolutional neural network (CNN) and gated recurrent unit (GRU) for IIoT traffic classification. Bayesian optimization fine-tunes hyperparameters. Experiments on combined, single, and cross-datasets show the superiority of the framework. The CNN-GRU model achieved 97.68% accuracy, 97.70% recall, 97.67% precision, and 97.68% F1-score for binary classification on the combined dataset. Transfer learning improved accuracy from 50.95% to 97.80% and F1-score from 48.52% to 97.79% when trained on X-IIoTID and tested on Edge-IIoTSet. Multi-classification accuracy and weighted averages of recall, precision, and F1-score were approximately 98.85%. [ABSTRACT FROM AUTHOR]

Abnormal ears (containing phenotypic differences in seeds, particularly in color) are manually removed to improve seed purity during seed production in the field or factory. Traditional convolutional neural networks (CNN) have significant parameters and greater network depth, making them unsuitable for deployment in resource-constrained embedded devices. This paper proposes a deep learning model (CornNet) based on custom lightweight CNN and improved training strategies for corn ears classification to address this issue. We improved the structure of VGG16 by reducing the number of convolution layers (Conv) and its channels to change network depth and used the global average pooling layer (GAP) instead of the fully connected layer (FC) to achieve a lightweight model. The Squeeze-and-Excitation network (SE) and Batch Normalization (BN) were used to improve the feature extraction ability and prevent gradient disappearance. The image acquisition environment, similar to the production line, was constructed to obtain images with consistent features to reduce the data required for training. Two training strategies (i.e., data augmentation and dynamic learning rate) were optimized to improve performance. The results showed that CornNet performed well compared to MobileNet, ShuffleNet, VGG16, ResNet50 and AlexNet in terms of accuracy, F1-score, model size, and FLOPs of 98.56 %, 98.93 %, 0.42MB and 0.07G, respectively. The improved training strategies improved accuracy by 3.07% to 16.08% and 0.26% to 30.91%. The CornNet proposed in this paper achieved a good balance between performance and computational cost, and it can obtain better generalization ability on small datasets than the traditional deeper networks model. • Proposed a seed corn ears classification model achieving a balance between the performance and computational cost. • Constructed an application scene similar to the practical application to obtain ears images. • Developed a comprehensive data augmentation based on offline and online augmentation. • Determined appropriately the initial learning rate and decay epochs of the dynamic learning rate. • The relationship between network depth and dataset size was analyzed. [ABSTRACT FROM AUTHOR]

Recently, surrogate models based on deep learning have attracted much attention for engineering analysis and optimization. Since constructing data pairs in most engineering problems is time-consuming, data acquisition is becoming the predictive capability bottleneck of most deep surrogate models, which also exist in surrogate for thermal analysis and design. In contrast with data-driven learning, enforcing the physical laws in building surrogates has emerged as a promising alternative to reduce the dependence on annotated data. This paper develops a physics-informed convolutional neural network (CNN) for the thermal simulation surrogate without labeled data. Firstly, we leverage the finite difference method to integrate heat conduction equation and loss function construction, guiding surrogate model training to minimize the violation of physical laws. Since the solution is sensitive to boundary conditions, we properly impose hard constraints by padding in the Dirichlet and Neumann boundaries. The proposed network can learn a mapping from heat source layout to the steady-state temperature field without labeled data, which equals solving an entire family of partial difference equations (PDEs). Moreover, the neural network architecture is well-designed to improve the prediction accuracy of the problem at hand, and pixel-level online hard example mining is proposed to overcome the imbalance of optimization difficulty in the computation domain, which is beneficial to the network training of physics-informed learning. The experiments demonstrate that the proposed method can provide comparable predictions with numerical methods and data-driven deep learning models. We also conduct various ablation studies to investigate the effectiveness of the proposed network components and training methods in this paper. Furthermore, the developed methods can be applied to other design and optimization applications which need to solve parameterized PDEs. [ABSTRACT FROM AUTHOR]

Skin cancer is the most prevalent type of cancer worldwide. Early detection is essential as it could be fatal at later stages. The classification of skin lesions is challenging since there are many variations, including changes in color, shape, size, high intra-class variation, and high inter-class similarity. In this paper, a unique class-wise attention method is proposed that considers each class equally while extracting additional discriminative information of skin lesions. The proposed attention mechanism is employed in a progressive manner to incorporate discriminative feature information from multiple scales. The proposed approach obtained competitive performance against more than 15 state-of-the-art methods including HAM1000 and ISIC 2019 leaderboard winners. The proposed method achieved 97.40% accuracy on the HAM10000 and 94.9% accuracy on the ISIC 2019 dataset. • A novel attention based mechanism is proposed for skin lesions classification. • The problem of significant intra-class variance, high inter-class similarity, and class imbalance is addressed by a class-wise attention mechanism. • The important features for the classification are extracted without the burden of additional parameters using a progressive class-wise attention mechanism. • The final three layers of the baseline model are discarded and the global average pooling (GAP) and classification output layers are added to reduce the number of parameters, while maintaining performance. • The proposed network is robust, end-to-end trainable and has good generalization on unseen data. [ABSTRACT FROM AUTHOR]

The antinuclear antibody (ANA) test plays the role of screening, diagnosis, treatment and prognosis for autoimmune diseases. However, the ANA test is time-consuming, and strongly depends on the interpretation of experienced medical examiners. With the rise of artificial intelligence, a lot of convolutional neural networks (CNN) have been used to recognize the features of ANA images, such as GoogLeNet, ResNet, and MobileNet. By extracting the disease-related features from the entire ANA image, these models achieve acceptable accuracy, but none of them can replace the manual ANA test completely. This is because they ignore the key features of ANA images. Accordingly, this paper aims to design an object detection model that can directly identify the key features of ANA images, i.e., the Metaphase cells. This process is critical in the manual ANA test performed by the medical examiners. Based on the YOLO model, our design first applies various image preprocessing and data augmentation techniques to enhance the critical features in the training data. We modify the original YOLO model, in which the residual block is replaced with SENet module to improve the classification performance by emphasizing critical features and reducing the influence of low-level features. Moreover, we use the different activation function to prevent the modified model from partial judgment. The experimental results show that our design can effectively improve the performance of ANA metaphase identification. Compared to the related CNN models, our design can improve F1-score and mAP by range 5%–14% and 11%–13%, respectively. In particular, the Kappa value of our design is 0.82 that is almost perfect agreement with the ANA experts. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT) has experienced phenomenal expansion over the past few years and has emerged as one of the most dynamic sectors of the international market. Android has become the driving force behind the rapid development of the IoT, and malware is one of the most severe concerns. Android is an open-source platform with a significant number of users. The identification of Android malware has become more critical in cybersecurity due to the meteoric rise of Android malware over the past few years and the severity of the damage it inflicts on smartphone users. Existing traditional Android malware detection methods based on machine learning have achieved encouraging performance. Still, the solution requires a massive workforce of feature engineering to build dynamic or static features and continuous learning through pre-extracted features to maintain high performance in identifying malware. Therefore, technologies for end-to-end malware detection without human interaction are necessary. In response to the rapid evolution of Android malware, a convolutional neural network that can acquire knowledge without requiring feature extraction is ideally suited. This paper offers a unique classification approach for malware based on a convolutional neural network, with feature extraction selected from the current pre-trained models DenseNet169, Xception, InceptionV3, ResNet50, and VGG16. The proposed methods have an advantage over existing detection methods due to their end-to-end learning process. Our suggested approaches convert Android APK files to binary codes and RGB images for usage as inputs to deep learning models. These models are trained and evaluated using the CICInvesAndMal2019 dataset, which consists of 333 benign and 428 malicious applications. Experiments demonstrate that the proposed approaches achieve a classification accuracy of 95.24%, 95.24%, and 95.83% for DenseNet169, InceptionV3, and VGG16, respectively, on a 761-sample dataset of malware, which is superior to other compared algorithms. Compared to previous methods, our suggested methods are more applicable to Android IoT devices since they do not require manual feature engineering and consume fewer resources. [ABSTRACT FROM AUTHOR]

Negative emotion detection is challenging during the peak time of the COVID-19 period. Most earlier studies contain individuals' physical health recognition rather than mental health detection, which is a significant concern in facing the COVID-19 situation. Identifying mental health in advance is essential to understand individuals' psychological condition. This paper considers the texts from social media during the pandemic of COVID-19. We propose a novel context-based auto-regressive transformer with bidirectional long short-term memory and a convolutional neural network (Context-ABT-BiLSTM-CNN) model to detect emotions such as abuse, anger, anxiety, depression, disgust, fear, guilt, sadness, and shame on social media. The existing works do not suggest relevant terms to detect suitable context; as a result, there is no scope for detecting emotions. We introduce a novel topic-based text (TBT) with a rule-based permutation (RBP) procedure to extract the relevant text from social media to identify emotions. Random search is suggested to store each input's correlated information and the order of each sequence. We recommend various transformer components to maintain the text sequence, avoid discrepancies during model training, capture the long-distance semantics in bidirectional contexts, and adopt both the permutation and factorization processes to build the model. Moreover, a comparative study is introduced to detect the most dominant emotions on social media during the pandemic and non-pandemic periods. The proposed model with XLNet embeddings surpasses state-of-the-art models for detecting text emotions. The ablation study is conducted to understand the essential components needed for the proposed model. [ABSTRACT FROM AUTHOR]

One of the most important features of a reliable power system is its capability to supply the demand continuously. This continuous supply has been maintained by the transient stability of the system against large disturbances. The study of this type of stability is examined through different indicators. One of the common indicators to evaluate the transient stability of the power system is the well-known Critical Clearing Time (CCT) index. Conventional methods for calculating CCT have presented good accuracy, however, their computational cost is very high which makes them not suitable for real-time applications and real large-scale networks. Considering their ability to feature extraction of big data, deep neural networks can be utilized as reliable tools to cover these deficiencies. In this regard, to cover the shortcomings of the conventional methods, this paper proposes a method based on deep Convolutional Neural Networks (CNN) to estimate the CCT index in real-time power system applications. Moreover, to analyze a realistic case, nonlinear limiters of the excitation systems which have a considerable effect on transient stability index are considered in this study. Thanks to the using of several deep layers and the comprehensive established database, the accuracy of proposed method is appropriately high. Numerical studies on IEEE standard networks as well as a real case in Iran Power Grid (IPG) represents the advantages of the proposed method. • A deep-learning-based convolutional neural network is designed to estimate CCT index of power system. • Nonlinear limiters of the excitation system are taken into account. • A real case study in Iran grid is considered for simulation. • As a result, the transient stability of system is accurately determined in on-line manner. [ABSTRACT FROM AUTHOR]

Identifying targets of LAser Detection And Ranging (LADAR) point clouds that are affected by noise and different poses throws a significant challenge. This paper proposes a novel LADAR point cloud target classification method that is both noise-robust and rotation-invariant. Specifically, the proposed method transforms the point cloud target into a slice image that is resilient to noise and holds rotation-invariance. Next, a designed 2D Convolutional Neural Network (CNN) is utilized to classify the corresponding point cloud target based on the slice image. To overcome the challenge of lacking prior knowledge about the discriminative features of the slice image, the 2D CNN is designed with the Local Importance-based Pooling (LIP) layer. This layer extracts the discriminative feature in a data-driven manner, thereby improving the accuracy of the classification process. The proposed method is evaluated through multiple experiments on the public ModelNet40 dataset. The experimental results demonstrate that the designed LIP-CNN can better learn the discriminative features of the slice image, achieving high classification accuracy. Moreover, the proposed slice-image-based method is capable of accurately classifying the target, even in the presence of noise and different poses. • Method for classifying LAser Detection And Ranging (LADAR) point cloud target. • Noise-robustness and rotation-invariance slice image for representing 3D target. • Data-driven method for keeping discriminative features of slice images. • Deep learning method for identifying slice images to classify 3D targets. [ABSTRACT FROM AUTHOR]

Gearboxes operate in harsh environments. Cloud-based techniques have been previously adopted for fault diagnosis in Gearboxes. Cloud-based fault diagnosis methods are prone to time delays and loss of information. Therefore, edge computing-based fault diagnosis becomes an option. However, with limited hardware resources for edge devices, balancing the diagnostic capabilities of the model with operating performance becomes a challenge. This paper proposes a lightweight convolutional neural network for gearbox fault diagnosis in edge computing scenarios to achieve an accurate diagnosis and lightweight deployment of models. By constructing the Mel-Frequency Cepstral Coefficients (MFCC) feature matrix of input data, the methodology can suppress noise interference and improve diagnostic accuracy. Based on the structural re-parameterization, the model structure transforms from multiple branches at training time to a single branch at inference time. This improves the inference speed of the model and reduces the hardware cost when the model is deployed while ensuring that the diagnostic capability of the model remains unchanged. Validation experiments were conducted on a public dataset and a custom experimental device, using the NVIDIA Jetson Xavier NX kit as the edge computing platform. According to the experiment result, after extracting the MFCC feature matrix, the average diagnostic accuracy rate in the noisy environment of the presented methodology is improved by 12.22% and 9.44%, respectively. After structural re-parameterization, the Memory of the model decreases by 52.58%, and the inference speed is increased by 38.83%. • A method for gearbox fault diagnosis in edge computing scenarios is proposed. • The inference speed and hardware overhead of the model can be highly improved through structural re-parameterization. • MFCC feature matrix is used to enhance the feature extraction capability of the network. • Two case studies show the effectiveness and efficiency of the proposed approach. [ABSTRACT FROM AUTHOR]

Convolutional neural networks are now seeing widespread use in a variety of fields, including image classification, facial and object recognition, medical imaging analysis, and many more. In addition, there are applications such as physics-informed simulators in which accurate forecasts in real time with a minimal lag are required. The present neural network designs include millions of parameters, which makes it difficult to install such complex models on devices that have limited memory. Compression techniques might be able to resolve these issues by decreasing the size of CNN models that are created by reducing the number of parameters that contribute to the complexity of the models. We propose a compressed tensor format of convolutional layer, a priori, before the training of the neural network. 3-way kernels or 2-way kernels in convolutional layers are replaced by one-way fiters. The overfitting phenomena will be reduced also. The time needed to make predictions or time required for training using the original Convolutional Neural Networks model would be cut significantly if there were fewer parameters to deal with. In this paper we present a method of a priori compressing convolutional neural networks for finite element (FE) predictions of physical data. Afterwards we validate our a priori compressed models on physical data from a FE model solving a 2D wave equation. We show that the proposed convolutional compression technique achieves equivalent performance in the prediction error as classical convolutional layers with fewer trainable parameters(around 20%) and lower memory footprint. [ABSTRACT FROM AUTHOR]

Perceiving the future trend of Vessel Traffic Flow (VTF) in advance has great application values in the maritime industry. However, using such big data from the Automatic Identification System (AIS) for accurate VTF prediction remains challenging. Deep training networks can learn valuable features from extensive historical data. This paper proposes a new learning-based prediction network, improved Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) with similarity grouping, including three views. To effectively enable the training network to capture the temporal and periodic (i.e. a spatial attribute) change characteristics of VTF, the CNN and LSTM are employed to compose spatial and temporal views, respectively. Hence, the original one-dimensional data is transformed into a matrix (hour of the day ✕ day) to adapt the input of the proposed methodology. In practical applications, VTF of multiple adjacent target regions need to be predicted simultaneously, and the changes of VTF in different areas may influence each other. To explore their hidden relationships, the similarity grouping view aims to find the target area that exhibits the most similarity with the VTF change trend of the current research area. Furthermore, similar information is combined with the features generated from the other two views to obtain the prediction results. In summary, the new advantage lies in mining the spatiotemporal attributes of data and fusing the similarity information of adjacent regions. Comparative experiments with eleven other methods on realistic VTF datasets show that the proposed method demonstrates superior prediction accuracy and stability performance. • Multi-perspective systematic analysis of ship trajectory prediction research. • Comparative analysis of eleven methods to reveal their advantages and disadvantages. • Applicability of the eleven extracted methods under difficult circumstances. • New solutions to STP for future hybrid traffic of manned and unmanned ships. [ABSTRACT FROM AUTHOR]