Search

Natural gas pipeline leakage is a common safety hazard, which can have a great impact on the economy and the environment. This paper proposed a novel manifold learning-enabled feature extraction method for natural gas pipeline leakage diagnosis. Firstly, the natural gas pipeline working condition signal is decomposed and denoised by Variational mode decomposition (VMD). Secondly, the denoised pipeline signals were constructed into a form expressed by the Symmetric positive definite matrix (SPD) using the VMD reconstruction technique, and the geodesic distance measurement method was applied to the SPD matrix to make the data located on the SPD manifold. Then feature extraction is carried out by Local linear embedding (LLE) method based on asymmetric distance. Finally, pattern recognition of the features extracted in this paper by Support vector machine (SVM) can achieve 100% recognition accuracy. By enabling faster and more accurate leak detection, the method minimizes gas loss, as well as mitigating the environmental risks caused by this potent greenhouse gas. [ABSTRACT FROM AUTHOR]

The reliability of bolt connections significantly impacts the operational state and lifespan of industrial equipment. Vision-based noncontact methods exhibit high efficiency in bolt loosening detection. However, limited image features hinder measurement accuracy. To improve bolt loosening detection performance, this paper proposes a novel deep learning backbone, the high-resolution cross-scale transformer, to extract high precision keypoints for bolt three-dimensional model construction. Simultaneously, a monocular vision measurement model is established to get the bolt exposed length and evaluate the connection loosening state. The proposed backbone hybridizes the advantages of high-resolution architecture and transformer, realizing global information aggregation and fine-grained image details. A simplified module, dual-scale multi-head self-attention, is designed to reduce the computational redundancy caused by the implementation of high-resolution multi-branch architecture. In the experiment section, the high-resolution cross-scale transformer outperforms other keypoint detection baselines, achieving the top one performance with 91.6 average precision and 84.9 average recall. The monocular vision measurement model realizes a 0.053 mm error with a 0.028 mm standard deviation, satisfying the industrial implementation requirement. Additionally, the model is tested on different industrial situations and an additional outside dataset, indicating the model's robustness and actual environment adaptability. • A monocular vision measurement method for bolt loosening detection is proposed. • It is the first attempt to introduce the transformer mechanism in bolt keypoint detection. • A new keypoint detection backbone is proposed in this paper for bolt feature extraction. • The 3D exposed length of bolts is calculated with a monocular vision system. • The bolt loosening detection method is validated. [ABSTRACT FROM AUTHOR]

Over the past decades, deep learning techniques have attracted increased attention from various research and industrial domains aligned with the development of Industry Internet-of-Things(IIoT). Specifically, with the advantage of data-driven methods, industrial organizations are seeking novel proactive strategies supported by analytic models to guarantee the quality of their production by observing degradation or predicting failure ahead of the occurrence of the component or asset. Predictive strategies are expected to promise the influence of unnecessary maintenance interruptions and mitigate the consequence of that, hence, extending the remaining useful life of products. This paper conducts a survey of the utilization of deep learning technologies on engineering applications where they provide satisfactory solutions with respect to specific data types or input signals. 106 primary papers are reviewed on deep learning–driven approaches which mainly explore five of the most popular architectures in the application of predictive maintenance. The main content of this paper summarizes the common advantages of each architecture and, accordingly, points out their limitations, as well as describes the application scopes of fully connected deep neural networks, convolutional neural networks, stacked autoencoders, deep belief networks, and deep recurrent neural networks. Based on the technique discussion for each of them, we intend to provide a comprehensive understanding and guidance of the appropriate usage of deep learning architectures to devise an effective predictive maintenance strategy for the scientific and industrial developers whose expertise lies in the prior domain knowledge of multi-source isomerization data. Moreover, the main content demonstrated the summarization of the decisive factor by which the incremental stages of the approaches were determined, fundamentally including the dataset specification, feature extraction, and the integration of deep learning approaches. [ABSTRACT FROM AUTHOR]

Track geometry is one of the critical indicators of railway tracks' condition which requires continuous monitoring and maintenance over time. In this paper, a novel artificial intelligence (AI) based framework is proposed for railway track geometry inspection using vibration data collected from a dedicated measuring high-speed train. This AI-based anomaly track detection approach consists of two main stages. Firstly, a subset of features that best characterizes the track condition is defined. Several dynamic features from time domain data are extracted and importance scores are assigned to them, to determine the most effective subset for the purpose of track condition monitoring. Secondly, a data-driven based anomaly detection approach is developed to assess and identify track geometrical defects. In this stage, the acceleration responses collected from an in-service train traversing on a healthy track zone are employed as input into a One-Class Support Vector Machine (OCSVM) algorithm. The proposed algorithm defines the anomalies as relative changes to the historical behaviour. A comprehensive dataset from field measurements using a Société Nationale des Chemins de Fer Français (SNCF) Réseau IRIS320 highspeed train is used in this paper to implement the proposed approach. In addition, the impact of using different features and different locations/directions of the sensors on the accuracy of detecting geometrical defects is investigated. It is also shown that the OCSVM approach outperforms other algorithms based on Isolation Forest (IF), Local Outlier Factor (LOF), and Robust Mahalanobis Distance (RMD) in terms of recall, precision, and F1-score. The proposed anomaly detection approach has demonstrated a 12% increase in defect detection accuracy compared to the direct utilization of the raw acceleration response, which can facilitate track monitoring using in-service trains while providing cost-efficient maintenance in the future. [Display omitted] [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is a crucial detection technique extensively employed in various contexts demanding accurate identification of human actions. Currently, mainstream HAR approaches frequently rely on sequential data generated by wearable sensors, with a primary focus on extracting feature representations that align with various human activities. Nevertheless, complex and diverse human activities encompass various kinds of features and substantial variations in the features emphasized across different activities, making it challenging for current artificial intelligence methods to achieve accurate recognition. To address this challenge, this paper introduces a network framework called the Multi-Feature Combining Attention Neural Network (MFCANN). This framework incorporates diverse feature extraction, along with both local and global feature attention mechanisms. It is composed of stacked Multi-Feature Combining Attention Blocks (MFCAB) designed by us. In contrast to traditional convolutional methods, MFCAB parallelly stacks various convolutional components, enabling the extraction of a broader range of features from human activity data. Additionally, we propose the Intra-Module Attention Block (Intra-MAB) and the Inter-Module Attention Block (Inter-MAB), which simultaneously focus on local fine-grained features within module feature maps and global distinguishing features across module feature maps. This aims to achieve more targeted feature learning, reinforcing the network's ability to distinguish between different human activities. Abundant experimental results demonstrate that the proposed MFCANN in this paper outperforms current mainstream deep learning algorithms in HAR tasks, achieving recognition accuracies of 0.9813, 0.9324, and 0.9930 on the UCI-HAR, USC-HAD, and RealWorld datasets respectively. We release our code at https://github.com/RangerHeart/MFCANN. [ABSTRACT FROM AUTHOR]

Track geometry is one of the critical indicators of railway tracks' condition which requires continuous monitoring and maintenance over time. In this paper, a novel artificial intelligence (AI) based framework is proposed for railway track geometry inspection using vibration data collected from a dedicated measuring high-speed train. This AI-based anomaly track detection approach consists of two main stages. Firstly, a subset of features that best characterizes the track condition is defined. Several dynamic features from time domain data are extracted and importance scores are assigned to them, to determine the most effective subset for the purpose of track condition monitoring. Secondly, a data-driven based anomaly detection approach is developed to assess and identify track geometrical defects. In this stage, the acceleration responses collected from an in-service train traversing on a healthy track zone are employed as input into a One-Class Support Vector Machine (OCSVM) algorithm. The proposed algorithm defines the anomalies as relative changes to the historical behaviour. A comprehensive dataset from field measurements using a Société Nationale des Chemins de Fer Français (SNCF) Réseau IRIS320 highspeed train is used in this paper to implement the proposed approach. In addition, the impact of using different features and different locations/directions of the sensors on the accuracy of detecting geometrical defects is investigated. It is also shown that the OCSVM approach outperforms other algorithms based on Isolation Forest (IF), Local Outlier Factor (LOF), and Robust Mahalanobis Distance (RMD) in terms of recall, precision, and F1-score. The proposed anomaly detection approach has demonstrated a 12% increase in defect detection accuracy compared to the direct utilization of the raw acceleration response, which can facilitate track monitoring using in-service trains while providing cost-efficient maintenance in the future. [Display omitted] [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) is a crucial detection technique extensively employed in various contexts demanding accurate identification of human actions. Currently, mainstream HAR approaches frequently rely on sequential data generated by wearable sensors, with a primary focus on extracting feature representations that align with various human activities. Nevertheless, complex and diverse human activities encompass various kinds of features and substantial variations in the features emphasized across different activities, making it challenging for current artificial intelligence methods to achieve accurate recognition. To address this challenge, this paper introduces a network framework called the Multi-Feature Combining Attention Neural Network (MFCANN). This framework incorporates diverse feature extraction, along with both local and global feature attention mechanisms. It is composed of stacked Multi-Feature Combining Attention Blocks (MFCAB) designed by us. In contrast to traditional convolutional methods, MFCAB parallelly stacks various convolutional components, enabling the extraction of a broader range of features from human activity data. Additionally, we propose the Intra-Module Attention Block (Intra-MAB) and the Inter-Module Attention Block (Inter-MAB), which simultaneously focus on local fine-grained features within module feature maps and global distinguishing features across module feature maps. This aims to achieve more targeted feature learning, reinforcing the network's ability to distinguish between different human activities. Abundant experimental results demonstrate that the proposed MFCANN in this paper outperforms current mainstream deep learning algorithms in HAR tasks, achieving recognition accuracies of 0.9813, 0.9324, and 0.9930 on the UCI-HAR, USC-HAD, and RealWorld datasets respectively. We release our code at https://github.com/RangerHeart/MFCANN. [ABSTRACT FROM AUTHOR]

Remote sensing scene (RSS) image classification plays a vital role in various fields such as urban planning and environmental protection. However, due to higher inter-class similarity and intra-class variability, achieving accurate classification for RSS images poses a considerable challenge for current convolutional neural networks (CNNs)-based and visual transformer (ViT)-based methods. To address these issues, this paper proposes a novel dual-encoding method named master–slave encoding network (MSE-Net) from two perspectives of feature extraction and fusion. The master encoder, based on ViT, extracts higher-level semantic features, while the slave encoder, based on CNN, captures relative lower-level spatial structure information. Secondly, to integrate feature information from the two encoders effectively, this paper further develop two fusion strategies. The first strategy involves the auxiliary enhancement units (AEUs), which eliminates semantic divergence between the two encoders, enhances spatial context awareness of the slave encoder and promotes effective feature learning. The interactive perception unit (IPU), as the second strategy, facilitates interaction and integration of the two encoders' representations to extract more discriminative feature information. In addition, we conducted comparative experiments on four widely-used RSS datasets, including RSSCN7, SIRI-WHU, the aerial image dataset (AID) and NWPU-RESISC45 (NWPU45), to verify the effectiveness of MSE-Net. The experimental results demonstrate that MSE-Net achieved state-of-the-art (SOTA) performance across all the datasets. [Display omitted] [ABSTRACT FROM AUTHOR]

Quick response codes are widely utilized in commodity anti-counterfeiting, and their traceability via smartphones is an effective authentication method. However, quick response codes are susceptible to illegal duplication, and the images captured by smartphones are easily affected by changes in light and environmental noise, leading to unsatisfactory verification results. To address these issues, this paper introduces a novel approach by combining the squeeze-excitation attention module with bottleneck residual block. It presents a squeeze-excitation bottleneck residual network for printer source identification of quick response codes. The squeeze-excitation attention module pays more attention to the features that represent the printer attributes, reduces the interference of useless information, and has low computational consumption; while the bottleneck residual block has the advantages of few parameters, strong feature extraction capability, and good expandability. Thus, the performance can be improved effectively only with a small increasing in parameters. The experimental results verify that the proposed method achieves an accuracy of 98.77% under smartphone capture conditions, it outperforms other convolutional neural network-based methods in terms of identification accuracy. The deep learning model proposed in this paper can be generalized and applied to the printer source identification of paper content in the civil, criminal investigation and judicial fields. [ABSTRACT FROM AUTHOR]

Applications in Engineering: In industry, surface defect detection is crucial for improving product quality. However, there are many challenges in industrial inspection scenarios, such as interference from background noise, complex small-target problems, significant variations in target objects, and the problem of finding a balance between inspection speed and accuracy. To address the above problems, this paper proposes an industrial defect-aware network based on cross-layer semantic information guidance and contextual attention enhancement (IDP-Net). Specifically, IDP-Net has four different new features. The contribution of artificial intelligence: Firstly, to solve the industrial surface context and defect similarity problem, this paper proposes a Lightweight Local Global Feature Extraction Network (LLG-Net), unlike other methods, the effective combination of self-attention blocks and convolution blocks ensures gradual integration of global and local features across multiple layers, to improve the detection ability of targets with significant changes in scale, this paper designs a Multiscale Perceptual Feature Aggregation Network (MPA-Net), adequately fuses the shallow fine-grained information and the deep semantic information. Then, to enhance the connection between multi-scale semantic information, an adaptive cross-layer feature fusion module (ACFF) is proposed, which is novel in integrating the characteristics of multiple adjacent levels to help the model better capture the different scale characterisation of the target. Finally, a Region Attention Module (RAM) is proposed and introduced in the detector to enhance the attention to the critical regions around the target object. In particular, this paper proposes a new localisation loss function (MEIoU) that enhances the network's attention to objects at different scales. The experimental results show that 94.3%, 98.7% and 99.5% of mAP@.5 are obtained on steel, PCB and aluminium surface defect datasets, respectively, and 50 FPS is achieved, which is better than the current mainstream detectors and meets the demand of practical industrial production. [ABSTRACT FROM AUTHOR]

The core of data series classification and clustering is how to evaluate the similarity between time series. However, for one-dimensional time series, the vast majority of similarity measurement methods currently relied on one-dimensional information. Although some classifications of time series were achieved by the images transformed from time series, they mostly relied on a single imaging mechanism (time domain or frequency domain). In order to give a new idea for time series similarity measurement, this paper proposed a similarity measurement method of time series based on the fused images. According to recurrence plots (RP) and wavelet scalogram (WS), the original time series was first transformed into images. Then, RP and WS were fused into an image. Next, the feature in the image was extracted by ResNet-18. Finally, the similarity of the time series can be obtained by the Euclidean distance between the features extracted by ResNet-18. The proposed method was verified by the time series from UCR Time Series Classification/Clustering Homepage. The results showed that the similarity measurement of time series can be effectively improved by fusing the images obtained by different imaging mechanisms in the time domain and frequency domain. • This paper proposes a new similarity measurement method for time series based on the image fusion of different forms. • The time domain and frequency domain information of time series are represented by recurrence plots and wavelet scalogram. • The characteristics of the image fused by recurrence plots and wavelet scalogram are extracted by Convolutional Neural Network to measure the similarity of time series. • 30 sets of time series show the validity and superiority of the proposed method. • The reasons for the effectiveness of the proposed method are analyzed in detail. [ABSTRACT FROM AUTHOR]

The aging of transformer oil–paper insulation (OPI) contributes significantly to transformer failure and reduced service life. Therefore, reliable and precise evaluation of the aging status of transformer insulation condition is crucial in averting transformer failure. This article proposes a novel deep learning-based aging condition assessment technique of transformers by combining FDS (Frequency domain spectroscopy) and squeeze-and-excitation-enabled convolutional neural network (SE-CNN). An intrinsic product-based transformation using a Gramian angular field is employed to dimensionally expand the 1-D FDS signals acquired from OPI samples into 2D images. Each feature extraction channel in the SE-CNN-based aging diagnosis framework is assigned a specific weight, enabling the model to concentrate on the most relevant features while discarding the rest. Furthermore, the effect of moisture variation on the model is also considered under three different conditions to ensure the consequence of compensation for the effect of moisture on FDS. A comparison with traditional machine learning models has also been made to ensure the viability of the proposed methodology. Additionally, the model is interpreted by creating explanation weight plots of the feature coefficients using a local interpretable model-agnostic explanations (LIME) method in Python. The findings demonstrate that the novel aging diagnosis framework visually assesses the transformer's OPI system with the highest accuracy of 99.12% in the test set. Furthermore, it facilitates the automatic extraction of deep aging features, thus providing better generalization and convergence capability. [ABSTRACT FROM AUTHOR]

Fault diagnosis is an important technology for performing intelligent manufacturing. To simultaneously maintain high manufacturing quality and low failure rate for manufacturing systems, it is of great value to accurately locate the fault element, evaluate the fault severity and find the fault root cause. In order to effectively perform feature extraction for rotating machinery fault diagnosis, a robust discriminant latent variable manifold learning (RDLVML) algorithm is proposed in this paper. Specifically, RDLVML algorithm could effectively select the features of the high-dimensional fault data, and extract the low-dimensional fault features with better discrimination. Through the idea of discriminative learning, the accuracy of fault diagnosis was improved by increasing the aggregation of the same state feature samples and the difference between different classes of feature samples. A novel weighted neighborhood graph is proposed by constructing the q -Rényi kernel function, which could effectively suppress the interference of outliers and noise, and make the RDLVML algorithm more robust. Moreover, the fault diagnosis method of rotating machinery based on RDLVML is proposed in this paper. Through experimental verifications and comparisons with several classical feature selection algorithms, rotating machinery fault diagnosis could be more accurately performed by this method. [ABSTRACT FROM AUTHOR]

Voice Conversion (VC) can manipulate the source speaker's identity of speech signal to make it sound like some specific target speaker, which makes it harder for a human being or a speaker verification/identification system to trace the real identity of the source speaker. However, extracting features of the source speaker from converted audio is challenging since the features of the target speaker are dominant in the converted audio, which hinders the extraction of the features of the source speaker. In this paper, to extract features of the source speaker from audios processed by VC methods, a speaker filtration block is designed, which uses mask estimation to identify source speakers from manipulated speech signals by filtering out the features of the target speaker in converted audio. Extensive experiments are conducted to evaluate the effectiveness of the proposed model in tracing source speakers of audios converted by ADAIN-VC, AGAIN-VC, VQMIVC, and FREEVC. Experimental results demonstrate the effectiveness of the proposed model by comparing to competitive baselines in speaker verification/identification scenarios. Notably, it has good performance even when being applied to unknown VC methods. Furthermore, the experiments also show that, training audios generated by multiple VC methods can improve the performance on the traceability of the source speaker. [ABSTRACT FROM AUTHOR]

The acoustic levitation technology expands the possibility of non-contact mass measurements, avoiding contact contamination and loss, particularly for tiny objects. Current acoustic levitated object mass estimation methods focus on the mechanism models between object mass and oscillating frequency. However, the same object exhibits different oscillating behaviors in the acoustic field, especially for non-spherical objects, resulting in obvious variability and non-stationarity in oscillating signals, which limits the estimation accuracy and versatility of mass. To address these challenges, this paper proposes a novel multi-modal hybrid network (MMH-Net) method for accurate mass estimation of levitated objects. This method directly correlates the object's actual mass with the oscillatory signal. Firstly, to reveal the inherent patterns of the oscillating signals, the variational mode decomposition (VMD) is introduced to decompose the variable oscillating signals into different frequency mode sub-windows. Secondly, a global module and a feature refinement module are constructed to extract the long-term dependencies of the original oscillatory signals and refine the local regularity patterns of the sub-modal signals, respectively. In addition, a weighted fusion strategy is adopted to integrate global and refinement features better. The specially designed dual spectrum self-attention module can assign reasonable weights to information from different sources, facilitating the extraction of representative features and effectively suppressing negative migration. Experimental results demonstrate that the proposed network model performs outstandingly in acoustic levitated object mass estimation. The proposed method is suitable to mass measurements of spherical and non-spherical objects, further promoting the development of acoustic levitation systems in research and application fields. [ABSTRACT FROM AUTHOR]

Industry 4.0 is of great significance for the development of oil industry. One of the pivotal steps towards achieving oil Industry 4.0 is accurately mastering the oilfield production dynamics, especially the changes of process parameters. Among many process parameters, the accurate modeling of water cut is extremely critical and difficult. In this paper, under laboratory conditions, oil–water flows with various preset water cuts are simulated. A fluid sensor, equipped with eight concave-shaped conductive electrodes, is employed to capture multi-channel measurement data, continuously recording the oil–water flow process from multiple angles. Subsequently, a novel flow behavior-heuristic deep learning model, named FBHWC model, is developed to model the relationship between the measurement data and water cut, achieving water cut measurement. The FBHWC model is guided by the complex flow behaviors of oil–water flows and consists of two key modules. Particularly, the multi-level feature fusion module focuses on the high-order feature extraction and fusion of sensor measurement data, while the multi-scale measurement module uses fully convolutional design to achieve accurate measurement of water cut. Experimental results show that the FBHWC model has excellent performance in measuring water cut, with mean square error of 0.016%. All these open up a new venue for exploring industrial multiphase flows through combining multi-electrode sensor and deep learning. [ABSTRACT FROM AUTHOR]

Electroencephalography (EEG) motor imagery (MI) signals have recently attracted much attention because of their potential to communicate with the surrounding environment in a specific way without the need for muscular and physical movement. Despite these advantages, these signals are difficult to detect due to their low signal to noise (SNR) rate and non-stationary and dynamic nature. Convolutional neural networks (CNN) can extract appropriate spatial and temporal features without the need for separate feature extraction and classification steps. Clean input data for CNN significantly improves its performance, but sometimes, common interference occurs between multi-channel signals. This challenge, along with the noisy EEG signals, degrades the performance of CNN networks. This paper presents a channel selection method based on convolutional neural networks to obviate these challenges. The proposed shallow convolutional neural network (SCNN) is designed with temporal convolution and pointwise convolution (using 1 × 1 convolution) to select the best channel with minimal computational load. Following the channel selection step, the multi-layer fusion CNN model was used to classify the signals. Choosing suitable and relevant EEG channels for motor imagery tasks makes it possible to create appropriate inputs for the classification model so that the model can be improved in the end. For High Gamma Dataset and Brian-Computer Interface (BCI) Competition IV-2a, the following accuracies were obtained: 81.15% and 72.01%, which are higher than when the other channel selections such as Mutual Information, Sequential Feature Forward Selection (SFFS) and wrapper methods are used. [ABSTRACT FROM AUTHOR]

Pointer meters are widely used in various industries, and there is a growing demand for automatic and non-intrusive access to meter readings. The existing methods of automatically calculating meter readings involve complex processes which are time-consuming and have low automation performance. The paper presents the design of a Separate Dual-Selective Attention Mechanism (SDSAM) module and a Reading Correction Module (RCM). We also introduce the SDSM-DenseNet model, which utilizes dense connections to directly predict meter readings based on meter images. The SDSM-DenseNet model demonstrates high automation performance and reduces reliance on inherent characteristics of the meter. Compared to other attention mechanism models and feature extraction models, the SDSM-DenseNet model achieves lower error rates in calculating meter readings. Furthermore, when compared with representative methods for reading meters, the average error rate of the SDSM-DenseNet model is reduced by approximately 48%, while only requiring 0.021 s to predict a single image. [ABSTRACT FROM AUTHOR]

The anomaly detection of aeroengines faces significant challenges, including high noise, complex parameter correlations, and imbalanced data. Current methods primarily rely on the reconstruction error of autoencoders to isolate anomalies. However, such methods are ineffective in detecting minor anomalies, as the reconstruction error for minor anomaly samples closely resembles that of normal samples. To address this problem, this paper proposes an innovative Deep Autoencoder Anomaly Detection Model (DAADM) for the anomaly detection of aeroengines. DAADM consists of two sub-networks: Anomaly Score Calculation Network (ASCN) and Deep Feature Extraction Network (DFEN). Firstly, ASCN introduces the adversarial training strategy into deep autoencoders, ensuring training stability while effectively isolating minor anomalies. Secondly, DFEN extracts deep-level features from input samples, providing a different perspective from ASCN to enhance sample information extraction. The combination of ASCN and DFEN compensates for their respective deficiencies, comprehensively considering input sample information, thereby promoting the capability of anomaly detection. Finally, the proposed DAADM is validated on real an aeroengine dataset and a public dataset. On the aeroengine dataset, the accuracy, recall, and F1 score of DAADM exceed the state-of-the-art method by 4.73%, 4.79%, and 5.76%, respectively. It is noteworthy that experimental results have proven that DAADM has strong anti-noise capabilities. [ABSTRACT FROM AUTHOR]

Accurate prediction of remaining useful life (RUL) is very important for the maintenance of mechanical equipment. With the latest development in sensor technology and artificial intelligence, the data-driven prediction method of mechanical equipment RUL has been widely concerned. However, in the past, the research on data mining was not sufficient, and the method of extracting degradation features was single. In order to solve this problem, this paper proposes a new method to extract the degradation features of bearings. This method creatively combines image with entropy methods and uses the Symmetrized Dot Pattern (SDP) method and Composite Multiscale Permutation Entropy (CMPE) to extract degradation features. Through five evaluation indexes, the degradation feature extracted by the SDP-CMPE method is compared with five common degradation features in time domain. Finally, a simple Elman neural network is used to predict RUL. Through the case study of bearing data set, it is verified that the degradation features extracted by the SDP-CMPE method are superior in prediction accuracy and conservatism. [ABSTRACT FROM AUTHOR]

The minimization of milling wear from prolonged contact between a tool and a workpiece is crucial in modern manufacturing, where high precision, efficiency, and productivity are required. Therefore, many researchers have developed systems for monitoring milling wear in real-time. However, while the adoption of multiple sensor signals can enhance the training effectiveness of the model, the complex setup process, data extraction, and feature analysis make it challenging to practically apply it for wear monitoring on production lines. Therefore, this study integrates various machining parameters of machine tools and a single type of sensor signal to extract features related to tool wear. These features are utilized as training data for an artificial neural network (ANN) model. The paper proposes a tool wear prediction ANN model based on automatic hyperparameter tuning and transfer learning, incorporating the architecture of Inception, DenseNet, and Layer Normalization. The network structure enhances the training efficiency for target features by expanding and reusing features from a single sensor's data. The combination of automatic hyperparameter tuning with a general optimizer optimizes the ANN architecture to achieve the best tool wear prediction model. Lastly, the proposed model is compared with other models using metrics including Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Coefficient of Determination (R 2) to evaluate the model performance. In the results, the proposed model shows the best performance among all the comparing models in all three metrics. The MAE is 0.03527, RMSE is 0.05135, and the R 2 is 0.9611. Compared to the common models such as Random Forest, the MAE and RMSE can be reduced by 43.2%, and 43.5% respectively, and a 9.5% R 2 performance increase. • The deep learning model is used for extended feature extraction for the proposed network for transfer learning. • This study proposes a fully connected layers structure which is similar to DenseNet with Layer Normalization at the end of each layer of dense connection. • The model optimization method is used to automatically adjust and optimize hyperparameters. • The experimental results show the proposed model which is capable of predicting tool wear in real-time by feeding the processing parameters has higher prediction accuracy than the other algorithms. [ABSTRACT FROM AUTHOR]

The detection of pomegranate fruitlets is of great significance for fruit thinning and yield estimation. However, the researches on the detection of pomegranate fruitlets are rarely considered in recent years. In addition, the immature fruit detection algorithms often have a large amount of calculation, which require high computing memory and computing resources of the equipment. Based on the mentioned issue, we propose a lightweight detection method based on the improved You Only Look Once Version 8(YOLOv8) to realize the automatic detection of pomegranate fruitlets (PG-YOLO). Firstly, the lightweight network Shufflenetv2 is used to reconstruct the backbone of YOLOv8s. Secondly, the Depthwise separable convolution is used to replace the standard convolution of Neck layer. Then, the multi-head self-attention (MHSA) is used to enhance the positioning ability and feature extraction ability of the model before the backbone network outputs features. Finally, the self-built pomegranate fruitlet dataset is used to train and test the PG-YOLO model. It is shown that, the Mean Average Precision(mAP) of pomegranate fruitlets detected by PG-YOLO model is 93.4%. The optimal weight size of PG-YOLO is only 2.2 Mb, which is 89.9% less than the original YOLOv8s. In addition, the detection speed of the PG-YOLO model is 74.1% higher than the original YOLOv8s. The results demonstrate that the study significantly reduces the complexity of the model and provides a research basis for further application in mobile devices. The self-built PG-YOLO-Dataset used in this paper has been published on github: https://github.com/LforikC/PG-YOLO-Dataset. [ABSTRACT FROM AUTHOR]

In this paper, we present a unified spatio-temporal attention MixFormer framework for visual object tracking. Within the vision transformer framework, we design a cohesive network consisting of target template and search region feature extraction, cross-attention utilizing spatial and temporal information, and task-specific heads, all operating in an end-to-end manner. Incorporating spatial and temporal attention modules within the network enables simultaneous feature extraction and emphasis, allowing the model to concentrate on target-specific discriminative features despite changes in illumination, occlusion, scale, camera pose, and background clutter. Stacking multiple non-hierarchical blocks allows meaningful features to be extracted while irrelevant features are discarded from the provided target template and search region. The simultaneous spatio-temporal attention module is employed to accentuate target appearance features and alleviate variation in the object state across frame sequences. Qualitative and quantitative analysis, including ablation tests based on various tracking benchmarks, validates the robustness of the proposed tracking methodology. [ABSTRACT FROM AUTHOR]

Deep learning has advanced the development of automated cervical cytology, yet limited studies have delved into methods for incorporating medical domain knowledge, and model interpretability has not been thoroughly investigated. To address this issue, this paper proposes a novel, explainable detection method for abnormal cervical cells, called dual-stream self-attention based feature fusion and origin grouping network (DSA-FFOGNet). To encourage the model to focus more on lesion cells and cell nuclei of diagnostic significance, the dual-stream self-attention (DSA) module is introduced to enhance the learning of lesion-specific features. In view of the complex background, cell dense distribution, cell overlap, or clumps existing in the actual cervical cytology images, multi-scale features are extracted and fused by using the path aggregation network (PAN) to enhance the feature representation ability. By integrating biomedical insights regarding cell provenance and formulating an origin grouping loss, DSA-FFOGNet adjusts the penalties for cervical cells originating from different groups, thereby enhancing the optimization of the model training process. To further improve the detection performance, the classification and localization tasks are decoupled via the use of double detection heads. Extensive experiments validate the robustness of the proposed DSA-FFOGNet. The visualization of class activation maps (CAMs) showcases the model's interpretability. The proposed approach advances the application and development of explainable artificial intelligence (XAI) models in cervical cytology and inspires further research in automated cervical cytology. [ABSTRACT FROM AUTHOR]

Abstract Cloud-type recognition is useful in preventing losses caused by adverse weather conditions. This paper presents a methodology and algorithms for automatic recognition of cloud types from color cloud images taken from the ground. We propose a number of eight algorithms for automatic cloud classification of seven cloud types defined by meteorological organization. Recognition is based on a number of features extracted from images which are related to color, texture, and shape. We introduce three new features based on the Fourier transform, namely, the modified k -FFTPX, the half k -FFTPX, and the h × k -FFT. The classification technique is based on artificial neural network (ANN) with tree algorithm to extract features. The proposed classification tree algorithm uses the technique called hierarchical classification which is composed of three levels of tree. Each level of the tree is capable of classifying up to four classes. We show that this method provides the highest accuracy at 98.08% through a series of four experiments. For accuracy assessment, each experiment splits dataset into train and test images using Leave-One-Out Cross-Validation (LOOCV). The result confirms that the hierarchical classification performs better than a single classification. In addition, the tree can be adapted to classify lesser number of cloud types. Our experiment reveals that the accuracy for classifying two classes, cloud and no-cloud, is high as 100%. Highlights • ANN based on tree algorithm is used for classifying 7 cloud types. • Three novel features are proposed based on Fourier transform. • Eight algorithms are presented in the paper. • Four experiments are conducted for testing ANN model on each level of tree. • Our algorithms provide a high accuracy of 98.08%. [ABSTRACT FROM AUTHOR]

Low probability of interception (LPI) radar is widely used in modern electronic warfare. With the increased radiation sources, multiple signals will arrive simultaneously. The traditional feature extraction method has too many features, which brings great trouble to the subsequent data processing. Most modulation recognition methods based on deep learning only consider the single signal after preprocessing, and the generalization ability is weak. This paper proposes a deep learning solution based on an improved you only look once version 8 (YOLOv8) network with a global attention mechanism (GAM), achieving recognition accuracy over 98% in a -10 dB signal-to-noise ratio (SNR) scenario to address the above problems. We improved the performance of the original network, which can be used in electronic countermeasures. [ABSTRACT FROM AUTHOR]

Malware often uses packing, an obfuscation strategy, to bypass antivirus. Identifying and understanding packers is therefore essential for analyzing suspicious binary files. Proposed machine learning methods for packer classification use a wide range of features, but many of them are redundant or irrelevant. It leads to waste of computational resources, so minimizing such features without reducing the effectiveness of packer identification is essential to analyze the ever-growing number of malware. This paper presents a novel embedded feature selection method for packer classification named Feature Selection based on Associative Classification (FSbAC). FSbAC exploits established concepts of association rule mining, particularly associative classification. As a result, FSbAC can define important sets of features per each packer individually, not just for the entire dataset. This makes it possible to directly learn the characteristics of each packer, improving the knowledge of analysts against packing. It also allows the costs associated with feature extraction to be taken into account during the selection process, ensuring the efficiency of the selected feature set. Seven different classification algorithms were used to assess the performance of FSbAC against eight FS methods. The evaluation encompassed different scenarios of packer classification, employing both a synthetic dataset and real-world datasets. The performance of the FS methods was evaluated with and without the inclusion of byte features vulnerable to bypass by hackers. Our results indicate that FSbAC is efficient for packer classification and leads to a substantial decrease in the number of features and in the computational resources required, without compromising predictive performance. • An embedded feature selection method based on associative classification was proposed. • Our method, named FSbAC, is focused on the packer classification problem. • We performed an extensive evaluation against 8 well-known feature selection methods. • FSbAC largely reduces the number of features without affecting predictive performance. • We emphasized interpretable classifiers to enrich the knowledge about packers. [ABSTRACT FROM AUTHOR]

Deep learning techniques have recently been utilized for model-free age-associated gait feature extraction. However, acquiring model-free gait demands accurate pre-processing such as background subtraction, which is non-trivial in unconstrained environments. On the other hand, model-based gait can be obtained without background subtraction and is less affected by covariates. For model-based gait-based age group classification problems, present works rely solely on handcrafted features, where feature extraction is tedious and requires domain expertise. This paper proposes a deep learning approach to extract age-associated features from model-based gait for age group classification. Specifically, we first develop an unconstrained gait dataset called Multimedia University Gait Age and Gender dataset (MMU GAG). Next, the body joint coordinates are determined via pose estimation algorithms and represented as compact gait graphs via a novel part aggregation scheme. Then, a P art- A dapt I ve R esidual G raph C onvolutional N eural Network (PairGCN) is designed for age-associated feature learning. Experiments suggest that PairGCN features are far more informative than handcrafted features, yielding up to 99% accuracy for classifying subjects as a child, adult, or senior in the MMU GAG dataset. These results suggest the feasibility of deploying Artificial Intelligence-enabled solutions for access control, surveillance, and law enforcement in unconstrained environments. [ABSTRACT FROM AUTHOR]

Existing neural network (NN) models that predict finite element analysis (FEA) of 3D trusses are not generalizable. For example, if a model is designed for a ten-bar truss, it cannot accurately predict the analysis results of a 12-bar truss. Such changes require new sample data and model retraining, reducing the time-saving value of the approach. This paper introduces Generalizable Surrogate Models (GSMs) that use a set of feature descriptors of physical structures to aggregate analysis data from various structures, enabling a more general model that predicts performance for a variety of geometric class, topology and boundary conditions. The paper presents training of generalizable models on parametric dome, wall, and slab structures, and demonstrates the accuracy and generalizability of these GSMs compared to traditional NNs. Results demonstrate first how to combine and use analysis data from various structures to predict the performance of the members of structures of the same class with different topology and boundary conditions. The results further demonstrate how these GSMs more closely predict FEA results than NN models exclusively created for a specific structure. The methodology of this study can be adopted by researchers and engineers to create predictive models for approximation of FEA. [ABSTRACT FROM AUTHOR]

This paper presents three texture features, viz., topothesy-fractal dimension, Hanman transform, and structure function based transform for the multispectral palmprint based authentication. It introduces the notion of information set originating from the Hanman–Anirban entropy. Using information set, Hanman transform features are derived. The topothesy-fractal dimension features arise from the structure function on representing the intensity variation on the texture surface. The structure function based transform features are derived from both structure function and the Hanman transform. Apart from the feature extraction, the fuzzy classifier based on the information processing is also developed. A novel score level fusion is proposed using Triangular-norms and Triangular-conorms. Thus the paper’s contribution is three-fold: i) New features for multispectral palmprint, ii) novel classifier for authentication, and iii) score level fusion for improving the accuracy. The rigorous experimental results certify that the proposed approaches make a substantial improvement in the authentication accuracy and outperform the contemporary approaches. [ABSTRACT FROM AUTHOR]

Link prediction is a fundamental problem in complex network analysis, aimed at predicting missing or forthcoming connections. Recent research investigates the potential of Nonnegative Matrix Factorization (NMF) models in reconstructing sparse networks and using deep NMF to uncover the hierarchical structure. Deep models have demonstrated remarkable performance in various domains, but they are susceptible to overfitting, especially on the limited training data. This paper proposes a novel Link Prediction using Adversarial Deep NMF (LPADNMF) to enhance the generalization of network reconstruction in sparse graphs. The main contribution is the introduction of an adversarial training that incorporates a bounded attack on the input, leveraging the L 2 , 1 norm to generate diverse perturbations. This adversarial training aims to improve the model's robustness and prevent overfitting, particularly in scenarios with limited training data. Additionally, the proposed method incorporates first and second-order affinities as input to capture higher-order dependencies and encourage the extraction of informative features from the network structure. To further mitigate overfitting, a smooth L 2 regularization is applied to the model parameters. To optimize the proposed model effectively, we utilize a majorization-minimization algorithm that efficiently updates the perturbation and latent factors in an iterative manner. Our findings demonstrate that the proposed model not only has the ability to uncover and learn complex structures but also possesses generalization capabilities. This method demonstrates superior performance compared to state-of-the-art methods across eight networks. In comparison to the best-performing approach, LPADNMF exhibited improvements of 2.36% in AUC, 4.67% in Precision, 2.21% in Recall, and 3.73% in F-Measure. [ABSTRACT FROM AUTHOR]

With the rapid expansion of the electric vehicle market, the demand for lithium-ion batteries (LIBs) is exploding. The state of health (SOH) of LIBs is receiving more widespread attention, which is the key parameter for battery health management. This paper proposes a SOH estimation method for LIBs via feature optimization and convolutional neural network (CNN) to reduce the information redundancy of existing multiple features, aimed at leveraging multiple sources of features while optimizing their combination to minimize redundancy. Firstly, multiple features are extracted from different perspectives, including electrical, thermodynamic, and electrochemical properties, to comprehensively characterize the aging of batteries. Secondly, we construct a SOH estimator based on principal component analysis (PCA) with CNN (PCA-CNN). Finally, the dimension of features is optimized with a simulated annealing algorithm (SA) under the mean-variance objective function. Moreover, Comparative experiments are conducted on the Oxford dataset for validation. The results demonstrate the effectiveness of the proposed multi-feature description method in terms of accuracy and smoothness. Compared to traditional CNN methods and fixed-dimension PCA-CNN, this estimation approach significantly improves performance, showing more than 20% and 30% increases in the key metrics of MAE and RMSE, respectively. This study successfully optimized feature combinations to reduce redundancy within the feature set while enhancing the accuracy of SOH estimation. [ABSTRACT FROM AUTHOR]

Due to the inevitable light absorption and scattering, underwater images always suffer from severe quality degradation, leading to significant performance decline for various maritime engineering-related applications. In recent years, deep neural networks (DNNs) have been proven to achieve high-quality enhancement of underwater images, which aims to extract and learn abstract features to exhibit superior performance. However, existing solutions often stack multilayer processing units to enrich features, which not only significantly burdens computational load but also encounters difficulties in fully utilizing and interacting between features at different depths. To address the above problems, this paper presents a cost-effective underwater image enhancement network via cascaded feature extraction, termed CFENet. Specifically, we develop a cost-effective cascaded structure for sufficient feature extraction and multiscale establishment of pixel-level long-range dependencies. In addition, we construct a dual-branch structure for effective feature fusion, so that the detailed texture and semantic information in the image can be simultaneously enhanced. Extensive experiments reveal the superiority of our proposed CFENet in both underwater image enhancement effects and computational complexity. Sufficient ablation study is conducted to demonstrate the effectiveness of each component in the network. [ABSTRACT FROM AUTHOR]

Dynamic facial expression recognition (DFER) is of great significance in promoting empathetic machines and metaverse technology. However, dynamic facial expression recognition (DFER) in the wild remains a challenging task, often constrained by complex lighting changes, frequent key-points occlusion, uncertain emotional peaks and severe imbalanced dataset categories. To tackle these problems, this paper presents a depth neural network model based on spatial key-points optimized region feature fusion and temporal self-attention. The method includes three parts: spatial feature extraction module, temporal feature extraction module and region feature fusion module. The intra-frame spatial feature extraction module is composed of the key-points graph convolution network (GCN) and a convolution network (CNN) branch to obtain the global and local feature vectors. The newly proposed region fusion strategy based on face spatial structure is used to obtain the spatial fusion feature of each frame. The inter-frame temporal feature extraction module uses multi-head self-attention model to obtain the temporal information of inter-frames. The experimental results show that our method achieves accuracy of 68.73%, 55.00%, 47.80%, and 47.44% on the DFEW, AFEW, FERV39k, and MAFW datasets. Ablation experiments showed that the GCN module, fusion module, and temporal module improved the accuracy on DFEW by 0.68%, 1.66%, and 3.25%, respectively. The method also achieves competitive results in terms of parameter quantity and inference speed, which demonstrates the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In practical engineering scenarios with limited labeled samples, conventional semi-supervised diagnostic methods face challenges in achieving satisfactory identification outcomes. To address the aforementioned issues, this paper introduces an incremental semi-supervised learning (ISL) approach based on boosting efficient attention (BEA) assisted cyclic adversarial auto-encoder (CAAE), referred to as BEA-CAAE. The CAAE enhances unsupervised feature representation by simultaneously constraining the distribution of encoded features and aligning elements of the reconstructed samples through a cyclic encoding strategy. The BEA improves classical attention weights' activation strength to better capture vital information, thereby boosting the feature extraction capabilities of both CAAE and the classifier. The ISL employs a stepwise pseudo-label propagation strategy to incrementally filter high-confidence samples, enhancing sample and label utilization, and improving diagnostic accuracy under low label rate conditions. Experiments conducted on multiple test rigs with simple structures as well as large-scale rotating components test rigs that mimic real-world engineering conditions have demonstrated that the proposed method exhibits a significant advantage over existing semi-supervised fault diagnosis approaches in terms of fault diagnosis accuracy and generalization capability, especially under low label rate conditions. [ABSTRACT FROM AUTHOR]

Longitudinal impulse, as a common dynamic problem of heavy haul trains, could induce the lateral instability of the coupler and the separation of the locomotive and wagon connection and thus seriously hinder the train running safety. The coupler force undergoes complex changes during longitudinal impulse transmission and exhibits strong nonlinear characteristics, which poses significant challenges to the identification of the coupler force. To address these issues, this paper reports a novel quantitative method for the coupler force identification of heavy haul locomotives. Firstly, the moving average filtering method is used to preprocess the signal of longitudinal relative displacement between the coupler and carbody (LRDCC), and the approximate relative velocity (ARV) feature is also extracted from the preprocessed signal. Then, a Convolutional Neural Network (CNN) model is constructed to extract high-level features from LRDCC and ARV. The model with randomly initialized weight parameters does not require additional training and thus reduces the computational cost. Finally, an Extreme Learning Machine (ELM) is used as the regression layer to process the extracted high-level features and further improve the identification performance. Additionally, the dung beetle algorithm is used to obtain the optimal model parameters to improve the coupler force identification performance. Not only a dynamics simulation but a field test is carried out to show that the proposed method could accurately detect the coupler force of heavy haul locomotives. Furthermore, the proposed method shows promising prospects in replacing the traditional monitoring method under the actual long-term working conditions of trains. • An indirect measurement scheme is proposed for coupler force dynamic identification. • The proposed network model presents high identification Accuracy and efficiency. • Method showcases strong generalization under complex, long-term train operations. [ABSTRACT FROM AUTHOR]

A data-driven approach for classifying defect patterns in wafer maps is crucial for automating the quality assurance process during semiconductor fabrication. Existing works have proposed machine learning-based methods to accomplish this important problem. However, many of these approaches assume model training from a supervised learning perspective, in which a substantial number of annotated wafer maps, each with its actual defect pattern, need to be available. Due to the cost and time required to generate such an annotated dataset, recently a few works started to explore the problem of identifying previously unseen defect patterns. However, the methods proposed in those works are able to only detect the presence of unseen defect patterns without identifying what those defect patterns are. To address this issue, this paper introduces a generalized zero-shot learning (GZSL)-based framework capable of classifying and identifying both seen and unseen defect patterns. It consists of a visual feature extraction phase based on semi-supervised contrastive learning (CL), a clustering phase using distance-based initialization, and an alignment phase based on the Wasserstein-Distance-based Visual Structure Constraint method. Our framework further reduces the need for expensive annotated datasets previously required to train supervised learning-based models, thereby improving the practicality of the data-driven approach for wafer map pattern classification. Furthermore, our framework introduces an effective way of incorporating additional semantic information in wafer map pattern classification that traditionally relied on training models solely from wafer maps. To the best of the authors' knowledge, our work is the first work to adopt GZSL in wafer map pattern classification. [ABSTRACT FROM AUTHOR]

The performance of object detection networks is often limited by the depth of the feature extraction network. Increasing network parameters may yield limited improvements in the detection system's performance. Additional careful designs of network details are necessary, but they can significantly increase training difficulty. This paper introduces a novel object detection method that utilizes generative adversarial training. Our approach focuses on minimizing the EM distance (Wasserstein distance) of the feature distribution as the primary training objective. We enhance the image features so that training of GAN (Generative Adversarial Networks) yields a feature distribution that exceeds that of the original dataset, obtaining a better feature extraction network. A new loss function is also added to the adversarial training process to ensure stable improvement of the detector. A comparative experiment conducted with the original CenterNet network on MS COCO (Microsoft Common Objects in COntext) 2017 reveals that the generative adversarial training method significantly improves the average precision for most of the examined backbone networks. Among the four backbone networks employed in the experiments, the mean improvement in network AP (Average Precision) values ranged from 0.3 to 0.9, demonstrating their success with minimal training efforts. Moreover, none of the four backbone networks experienced an increase in network parameters during inference. Experimental results indicate that the proposed architecture effectively enhances the network's feature extraction capability without compromising speed during inference. [ABSTRACT FROM AUTHOR]

The significant gap between the assumed and actual degradation model will undoubtedly lead to a serious performance decrease for deep learning based image super-resolution (SR) methods. To solve this shortcoming, this paper presents an enhanced blind image super-resolution (EBSR) network based on unsupervised degradation representation learning. The proposed EBSR mainly is made up of two branching subnetworks. The first subnetwork is a residual-based degradation encoder, which is responsible to learn a high-dimensional abstract degradation representation vector from the input low-resolution (LR) image patches using contrast learning. Another branch is the degradation-aware fusion SR (DAFSR) network that completes the image SR task with the aid of the degradation representation vector learned by the encoder. To obtain an improved SR performance under various degradation settings, the degradation-aware fusion block (DAFB), the core block of DAFSR, has been elaborated, which embeds the degradation model information learned by the encoder into every layer of the network. Additionally, a shallow feature extraction block (SFEB) is constructed for DAFSR to extract global information from the LR image. The proposed EBSR can flexibly adapt to various degradation models. Extensive experimental results on synthetic datasets and real images show that the proposed EBSR is able to achieve a leading reconstruction performance over other blind SR algorithms. • A boosted degradation representation learning is proposed for blind image SR. • A new degradation encoder is designed using residual connection. • A degradation-aware fusion block is designed to obtain improved SR performance. • A shallow feature extraction block is adopted to extract global informative features. [ABSTRACT FROM AUTHOR]

Group activity recognition aims to recognize holistic activity in multi-person scene, necessitating consideration of the interactions between actors and their surroundings. It has various applications, such as public surveillance, video analysis. Nonetheless, existing works merely extract scene features as a supplementary component of activity features, failing to adequately explore the interplay between the scene and actors. To address this limitation, this paper proposes a Local–Global Context-Aware Graph Reasoning Model (LG-CAGR), which leverages and reasons through local and global context features to gain deeper insights into group activity within the scene. In particular, we present an innovative feature extraction strategy to harness local location features and global scene attributes, effectively complementing actor features by capturing spatial group topology and determining relative positions between actors. Subsequently, we delve into these features by devising a local–global group reasoning module that deduces pair-wise interactions between actors and scenes within Graph Convolutional Network, comprehensively elucidating correlations between overall scene and local individuals to construct group-level features. Multi-graphs are constructed considering actor's features, scene features as nodes, and interactions as edges. A self-attention graph pooling network is introduced to automatically integrate key actor features and form rich group-level features to recognize group activity. The results on Collective Activity Dataset, Collective Activity Extended Dataset, Volleyball Dataset and Public Life in Public Space dataset have reached 94.0%, 97.7%, 92.7% and 56.1%. Compared with existing methods using the same backbone, we exceeded 1%, 2.1%, 0.3%, and 14.9% respectively, affirming the superiority of the proposed method compared with state-of-the-art methods. • A novel progressive network structure for group activity recognition. • Context feature extraction strategy for the exploration of group activity. • Capturing temporal and spatial dynamics by reasoning local–global GCN. [ABSTRACT FROM AUTHOR]

This paper delves into enhancing underwater images to improve human observation of underwater scenes. To this end, a reinforcement learning-based human visual perception-driven image enhancement paradigm for underwater scenes is proposed, referred to as INSPIRATION. This paradigm is so named because it constructs rewards for guiding model training by incorporating metrics inspired by human visual perception, eliminating the need for reference images. It overcomes the limitations of deep models, which often result in poor performance in dynamic and diverse underwater scenes due to the scarcity of underwater images with enhancement references. Specifically, a residual-enhancement network, composed of residual modules and a channel-attention module, is utilized to extract features as a state. An extensible collection encompassing diverse image enhancement algorithms is utilized to provide an image enhancement algorithm as an action. A multi-non-reference human visual perception metric increment is utilized as a reward, and proximal policy optimization (PPO) is utilized to conduct reinforcement learning. This paradigm treats underwater images as sole inputs during both training and implementation, learning and organizing a sequence of image enhancement algorithms to explicitly achieve a step-wise image enhancement process, aligning the enhanced images with human visual perception. Extensive qualitative and quantitative evaluations conclusively demonstrate that our paradigm outperforms nine state-of-the-art underwater image enhancement methods in terms of visual quality and achieves better performance on five underwater image quality assessment measures across three underwater image datasets. Additionally, the potential advantages of our paradigm as a pre-processing step in other underwater computer vision applications are demonstrated. The code has been released publicly for reproducibility and evaluations at https://gitee.com/wanghaoupc/uie_inspiration. [ABSTRACT FROM AUTHOR]

Semantic segmentation is a well-studied topic and one of the most challenging tasks in computer vision applications, such as autonomous driving. Deep learning approaches based on convolutional neural networks (CNN) have demonstrated exceptional success on this task in recent years. Despite this success, existing approaches are plagued by higher-order inconsistencies between the ground truth images and the ones predicted by the segmentation model. This paper proposes a novel post-processing scheme based on adversarial learning to counter these inconsistencies. Such a scheme can be combined with a variety of existing CNN-based semantic segmentation networks to improve their segmentation performances. The proposed scheme is a Two-Stream Conditional Generative Adversarial Network (TScGAN), with one stream having initial semantic segmentation masks predicted by an existing CNN, while the other stream utilizes scene images to retain high-level information under a supervised residual network structure. In addition, TScGAN incorporates a novel dynamic weighting mechanism, which leads to significant and consistent gains in segmentation performance. Several comparative tests on public benchmark driving databases, including Cityscapes, Mapillary, and Berkeley DeepDrive100K, demonstrate the effectiveness of the proposed method when used with state-of-the-art CNN-based semantic segmentation models. Furthermore, the ablation experiment proved the structural rationality of our two-stream structure. The code for TScGAN could be found at https://github.com/epan-utbm/TScGAN-for-Improving-Semantic-Predictions • Explores various post-processing techniques for enhancing semantic segmentation. • Novel Two-Stream Conditional GAN enhances urban drive scene segmentation. • Enhances feature extraction by implementing dynamic weighting within streams. • Evaluates the model against existing semantic segmentation methodologies. • Tests on Cityscapes, Mapillary, BDD100K reveal major IoU gains with proposed model. [ABSTRACT FROM AUTHOR]

Owing to the development of deep learning, indoor personnel detection methods based on deep neural networks have been extensively investigated in recent years. However, more complex and deeper network structures may consume more computational resources, which seriously limits the deployment of large-scale deep neural networks on lightweight devices. In view of this, a lightweight anchor-free one-level feature indoor personnel detection method based on transformer (LAOF-IPDT) is proposed in this paper, which is deployed on two embedded devices and achieves good detection accuracy. In the feature extraction backbone, an enhanced cross-stage partial ghost convolution block with ghost convolution and channel shuffle is designed to extract shallow features. Additionally, to obtain more comprehensive global features in the high-level semantic structure, an embedded vision transformer cross-stage partial block is constructed by embedding a lightweight mobile-friendly vision transformer. For the path-aggregation neck, a lightweight feature pyramid network is proposed, which integrates multi-scale feature maps to obtain richer one-level feature representations. Subsequently, a dilated convolution group block is applied to expand the one-level feature receptive field and detection is accomplished using a one-level feature map. For the detection head, an anchor-free mechanism is applied to reduce the hyper-parameter interference of anchor boxes. Extensive experiments on four datasets indicated that the LAOF-IPDT outperforms other lightweight networks in terms of accuracy, speed, model parameters, and model size. For example, the frames per second of the LAOF-IPDT are 8.39 and 14.67 on CPU devices and Jetson Nano devices, respectively, and the mean average precision is 84.49%. [ABSTRACT FROM AUTHOR]

Remaining Useful Life (RUL) prediction is of great significance for maintaining the reliability and safety of industrial equipment. To address the challenges faced by existing methods in simultaneously extracting local and global degradation information from monitoring data. This paper proposes a Two-Stream Convolution Augmented Transformer (TACT) model based on L 2 regularization constraint. Specifically, we design the parallel multi-scale Convolution Neural Network (CNN) and Transformer module to combine the local modeling ability of CNN and the global modeling ability of Transformer to improve the overall architecture of RUL prediction model. Moreover, the two-stream network based on the parallel structure also realizes the synchronous extraction of different time steps and sensor features in the sequence. Then, in the process of model training, the prediction reliability constraint is fused, the delay prediction constraint term is introduced, and the L 2 regularization loss function is constructed. Finally, extensive experiments on the commercial modular aero-propulsion system simulation (C-MAPSS) show that our model provides competitive performance in terms of Root-Mean-Square Error (RMSE) and Score metrics. Compared to the state-of-the-art method based on Recurrent Neural Network (RNN) or CNN and its variants, Score is reduced by at least 2.71% and RMSE by at least 3.13%. Compared to the Transformer-based improved method, the Score is decreased by at least 4.54% and the RMSE is decreased by at least 2.78%. The effectiveness of the proposed method is demonstrated. • A hybrid end-to-end RUL prediction network based on a tandem combination of CNN and Transformer is proposed. • A two-stream structure is used to extract spatial and temporal features simultaneously. • The MSE loss function incorporating L 2 regularization constraint is proposed. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]

The pivotal factor to regulate and enhance within an operating batch is the quality process, a challenging variable to monitor online. Soft sensors offer an immediate alternative for providing real-time insights into process quality, yet persisting issues include the imbalance between process and quality measurements, noisy measurements, and the intricate 3-dimensional dynamic batch data structure. This paper introduces the Robust Semi-Supervised Dual-Attention Latent Dynamic Conditional State-Space Model (RS2DA-LDCSSM) to address these challenges for within-batch quality prediction. Given the frequent absence of quality data due to measurement inconveniences, an imputation network is embedded within the RS2DA-LDCSSM to facilitate the prediction of future quality. To prevent information distortion during data unfolding, the Attentional Sequence-to-Sequence RNN Encoder-Decoder (AS2S-RNNED) is employed to process the 3-dimensional batch data. The proposed method integrates AS2S-RNNED with a probability state-space model to filter out noisy process data and stabilize the probability prediction of quality data, extracting spatial and temporal latent data from past process and quality data while minimizing the loss in multi-step prediction. This work represents a novel approach to dynamic nonlinear batch processes through probabilistic semi-supervised learning. The RS2DA-LDCSSM is adaptable to any batch process for within-batch quality prediction, as evidenced by numerical cases demonstrating its reliability, achieving an R2 value of 0.87, surpassing comparison methods. In an industrial penicillin fermentation batch process, the RS2DA-LDCSSM exhibits remarkable robustness with a high R2 index value of 0.99 in practical quality prediction scenarios, showcasing its efficacy. • Semi-supervised model training utilizes unlabeled process data. • Separate networks are crafted for process and quality feature extraction. • The state-space model with the encoder-decoder is used to extract latent features. • A multi-step-ahead prediction learning scheme yields favorable prediction results. [ABSTRACT FROM AUTHOR]

This paper delves into enhancing underwater images to improve human observation of underwater scenes. To this end, a reinforcement learning-based human visual perception-driven image enhancement paradigm for underwater scenes is proposed, referred to as INSPIRATION. This paradigm is so named because it constructs rewards for guiding model training by incorporating metrics inspired by human visual perception, eliminating the need for reference images. It overcomes the limitations of deep models, which often result in poor performance in dynamic and diverse underwater scenes due to the scarcity of underwater images with enhancement references. Specifically, a residual-enhancement network, composed of residual modules and a channel-attention module, is utilized to extract features as a state. An extensible collection encompassing diverse image enhancement algorithms is utilized to provide an image enhancement algorithm as an action. A multi-non-reference human visual perception metric increment is utilized as a reward, and proximal policy optimization (PPO) is utilized to conduct reinforcement learning. This paradigm treats underwater images as sole inputs during both training and implementation, learning and organizing a sequence of image enhancement algorithms to explicitly achieve a step-wise image enhancement process, aligning the enhanced images with human visual perception. Extensive qualitative and quantitative evaluations conclusively demonstrate that our paradigm outperforms nine state-of-the-art underwater image enhancement methods in terms of visual quality and achieves better performance on five underwater image quality assessment measures across three underwater image datasets. Additionally, the potential advantages of our paradigm as a pre-processing step in other underwater computer vision applications are demonstrated. The code has been released publicly for reproducibility and evaluations at https://gitee.com/wanghaoupc/uie_inspiration. [ABSTRACT FROM AUTHOR]

Group activity recognition aims to recognize holistic activity in multi-person scene, necessitating consideration of the interactions between actors and their surroundings. It has various applications, such as public surveillance, video analysis. Nonetheless, existing works merely extract scene features as a supplementary component of activity features, failing to adequately explore the interplay between the scene and actors. To address this limitation, this paper proposes a Local–Global Context-Aware Graph Reasoning Model (LG-CAGR), which leverages and reasons through local and global context features to gain deeper insights into group activity within the scene. In particular, we present an innovative feature extraction strategy to harness local location features and global scene attributes, effectively complementing actor features by capturing spatial group topology and determining relative positions between actors. Subsequently, we delve into these features by devising a local–global group reasoning module that deduces pair-wise interactions between actors and scenes within Graph Convolutional Network, comprehensively elucidating correlations between overall scene and local individuals to construct group-level features. Multi-graphs are constructed considering actor's features, scene features as nodes, and interactions as edges. A self-attention graph pooling network is introduced to automatically integrate key actor features and form rich group-level features to recognize group activity. The results on Collective Activity Dataset, Collective Activity Extended Dataset, Volleyball Dataset and Public Life in Public Space dataset have reached 94.0%, 97.7%, 92.7% and 56.1%. Compared with existing methods using the same backbone, we exceeded 1%, 2.1%, 0.3%, and 14.9% respectively, affirming the superiority of the proposed method compared with state-of-the-art methods. • A novel progressive network structure for group activity recognition. • Context feature extraction strategy for the exploration of group activity. • Capturing temporal and spatial dynamics by reasoning local–global GCN. [ABSTRACT FROM AUTHOR]

Remaining Useful Life (RUL) prediction is of great significance for maintaining the reliability and safety of industrial equipment. To address the challenges faced by existing methods in simultaneously extracting local and global degradation information from monitoring data. This paper proposes a Two-Stream Convolution Augmented Transformer (TACT) model based on L 2 regularization constraint. Specifically, we design the parallel multi-scale Convolution Neural Network (CNN) and Transformer module to combine the local modeling ability of CNN and the global modeling ability of Transformer to improve the overall architecture of RUL prediction model. Moreover, the two-stream network based on the parallel structure also realizes the synchronous extraction of different time steps and sensor features in the sequence. Then, in the process of model training, the prediction reliability constraint is fused, the delay prediction constraint term is introduced, and the L 2 regularization loss function is constructed. Finally, extensive experiments on the commercial modular aero-propulsion system simulation (C-MAPSS) show that our model provides competitive performance in terms of Root-Mean-Square Error (RMSE) and Score metrics. Compared to the state-of-the-art method based on Recurrent Neural Network (RNN) or CNN and its variants, Score is reduced by at least 2.71% and RMSE by at least 3.13%. Compared to the Transformer-based improved method, the Score is decreased by at least 4.54% and the RMSE is decreased by at least 2.78%. The effectiveness of the proposed method is demonstrated. • A hybrid end-to-end RUL prediction network based on a tandem combination of CNN and Transformer is proposed. • A two-stream structure is used to extract spatial and temporal features simultaneously. • The MSE loss function incorporating L 2 regularization constraint is proposed. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]