Search

[Display omitted] In today's world, a large number of people suffer from motor impairment-related challenges. Rehabilitation is the main method used to overcome these difficulties. The goal of the paper is to develop a deep learning-based electroencephalogram (EEG) sensor-controlled assistive device for the rehabilitation of elbow and finger movements. We have introduced an innovative finger and elbow movement rehabilitation method using an EEG sensor. The EEG sensor's recorded EEG signals, attention values, and meditation values have been used for this purpose. This rehabilitation technique helps a person perform basic finger movement rehabilitation motions, such as finger extension and flexion. Also, basic elbow movement rehabilitation exercises, i.e., elbow extension and elbow flexion, can be performed by using this rehabilitation technique. In this research, an EEG sensor records the prefrontal lobe's EEG signals, attention value, and meditation value of a person while the person performs motor imagery. A deep learning-based CNN-TLSTM (Convolution Neural Network-tanh Long Short-Term Memory) model with attention mechanism has been designed for decoding the EEG sensor recorded data. The trained deep learning model decides the course of action of the rehabilitation device. The designed model achieves an accuracy of 99.6%. A working prototype model of the rehabilitation device has been developed, and the overall success rate of the model is found to be 98.66%. The novelty of the paper lies in i) designing an attention-based CNN-TLSTM model for motor imagery classification and ii) developing a low-cost EEG sensor-driven rehabilitation device for finger and elbow movement rehabilitation. [ABSTRACT FROM AUTHOR]

When writing academic papers, referencing statements, claims and previous studies is always an important activity. However, it is considered challenging for scientists to find relevant and appropriate scientific articles which are closely related to their current works in order to reference them in their research. As a consequence of the rapid growth in scientific papers being published every year, researchers might easily get overwhelmed within a huge number of resources. One way to help them find the desired references more easily is to use context-aware citation recommendation. The citation recommender system can automatically provide a list of suitable papers as references based on specified inputs which reflect the researchers' interests. Among the outstanding achievements of deep learning and natural language processing in recent years, the utilization of deep neural learning architectures have supported to address the problem of citation recommendation. As the result, the neural citation recommendation area has received much attention from the academic community, with the aim of enhancing the precision and correctness of the results of existing citation recommendation systems. Following this research direction, in our paper we present a novel context-aware citation recommendation model, called RHN-DualLCR (Recurrent Highway Networks – Dual Local Citation Recommendation), which integrates Recurrent Highway Networks (RHN), an improved model of the original Bidirectional Long Short-Term Memory (BiLSTM) architecture, and uses a SciBERT-based (Science text of Bidirectional Encoder Representations from Transformers) embedding layer to build up the efficiency of the state-of-the-art local citation recommendation model, which enriches context representation with global information. Our research demonstrates its originality and relevance because we used one of the latest achievements of deep models (RHN model) and natural language processing (SciBERT) applied to the citation recommendation problem. We have conducted experiments on the RHN-DualLCR model on 3 widely known datasets for the citation recommendation problem: ACL-200 (Association for Computational Linguistics), ACL-600 and RefSeer, and also used 2 common evaluation standards Mean Reciprocal Rank (MRR) and the Recall@K (R@K for short) to evaluate the performance of our model. Experimental results show that our proposed model is 3% to 16% better than the original models or state-of-the-art models. [ABSTRACT FROM AUTHOR]

In the realm of ecological management and human activities within river basins, short-term runoff forecasting plays a pivotal role. Addressing this need, this paper introduces an innovative framework for short-term runoff forecasting: the Ensemble Attention Temporal Convolutional Network (EA-TCN). The cornerstone of this innovation lies in the effective amalgamation of Temporal Convolutional Network (TCN), lightweight attention mechanism, and ensemble learning strategy. This integration synergistically enhances the model's overall performance in terms of accuracy, efficiency, and robustness. TCN forms the foundation of this framework, where its efficient architecture, characterized by shared parameters and parallel computation, significantly boosts computational efficiency. Its employment of causal and dilated convolutions adeptly captures long-term dependencies within time series inputs. The incorporated lightweight attention mechanism further augments the TCN, enabling EA-TCN to precisely discern complex relationship in temporal data, particularly exhibiting remarkable temporal robustness across various forecasting horizons—a feat challenging for conventional forecasting approaches. Additionally, the integration of the Snapshot ensemble method within the framework allows for simulating the effect of training multiple models through one single training process, thus further elevating the model's accuracy and robustness. Rigorous ablation and comparative experiments conducted on the US Columbia River dataset substantiate our claims. The results not only validate the individual merits of each component within EA-TCN but also illuminate the significant advantages of their collective application. Our comprehensive assessment unequivocally demonstrates the framework's exceptional performance in short-term runoff forecasting, positioning it as a state-of-the-art solution in this field. We will further discuss its impact of vocation education in this industry. • This paper applied the modified TCN to short-term runoff forecasting. • EA-TCN adopts a lightweight plug-and-play attention module in the time dimension. • The Snapshot ensemble method is also applied to our proposed model. • EA-TCN can make accurate predictions for multiple different lead times. [ABSTRACT FROM AUTHOR]

Post-hoc explaining approaches for deep learning (DL) models has attracted much attention in safety–critical applications such as rotating machinery intelligent fault diagnosis (IFD). However, with the help of the explanation techniques, the models are still fragile to domain shifts caused by varying speeds and loads without help for improving their cross-domain performance. Since humans in the decision-making loop are essential for a reliable diagnostic system to determine the reliability of diagnosis, this paper proposes a causal explaining guided domain generalization (CXDG) method to realize trust worthy IFD with human in the decision loop. Specifically, an explaining model is trained with the conditional mutual information, which is a causal strength metric, and utilized to tell the causal features in the input data as the attributions of the diagnostic model. A translation process of the attributions is proposed to make the explaining process understandable. Furthermore, the aim of this paper is not only explaining but also beyond that the diagnostic model is guided to focus on the causal features to improve its generalization ability in unseen domains. The effectiveness of the method is validated on two experiment datasets. The results show that the proposed method can both explain the attributions of the diagnosis model and be beneficial to the generalization ability. [ABSTRACT FROM AUTHOR]

In order to realize the digital management and manufacturing of air material, the methods of 3D point cloud enhancement optimization and high-fidelity texture reconstruction in entity digitization of air material based on the fusion of 3D scanning and neural rendering are proposed. In this paper, an automatic data acquisition system is designed and built, and the acquired training images are enhanced. Then the Yolov8 segmentation model and CascadePSP boundary refinement model are combined for training to realize foreground segmentation and background removal of air material images. In this paper, a 3D point cloud enhancement optimization method is proposed, which combines the point cloud obtained by binocular structured light scanner and the point cloud reconstructed by Colmap algorithm for registration and fusion. By combining the coarse registration of PointNetLK network with the fine registration of ICP algorithm, a more complete and high-quality dense point cloud is generated, and the effects of illumination conditions and image quantity on point cloud reconstruction is studied experimentally. Using the optimized point cloud model and random perspective enhancement (RPE) method, the neural rendering model NeuS is improved, which improves the 3D surface texture reconstruction quality and the sight extrapolation effect. In order to verify the effectiveness of the proposed method, comparison and ablation experiments are carried out in this paper. The experimental results show that point cloud enhancement optimization and RPE have a good effect on improving the model, and the model performance has been greatly improved. [ABSTRACT FROM AUTHOR]

Transformers are Deep Neural Networks (DNN) that utilize a self-attention mechanism to capture contextual relationships within sequential data. Unlike traditional neural networks and variants of Recurrent Neural Networks (RNNs), such as Long Short-Term Memory (LSTM), Transformer models excel at managing long dependencies among input sequence elements and facilitate parallel processing. Consequently, Transformer-based models have garnered significant attention from researchers in the field of artificial intelligence. This is due to their tremendous potential and impressive accomplishments, which extend beyond Natural Language Processing (NLP) tasks to encompass various domains, including Computer Vision (CV), audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published, spotlighting the Transformer's contributions in specific fields, architectural disparities, or performance assessments, there remains a notable absence of a comprehensive survey paper that encompasses its major applications across diverse domains. Therefore, this paper addresses this gap by conducting an extensive survey of proposed Transformer models spanning from 2017 to 2022. Our survey encompasses the identification of the top five application domains for Transformer-based models, namely: NLP, CV, multi-modality, audio and speech processing, and signal processing. We analyze the influence of highly impactful Transformer-based models within these domains and subsequently categorize them according to their respective tasks, employing a novel taxonomy. Our primary objective is to illuminate the existing potential and future prospects of Transformers for researchers who are passionate about this area, thereby contributing to a more comprehensive understanding of this groundbreaking technology. • The paper presents a comprehensive survey on transformers for deep learning tasks. • The paper conducts a thorough analysis on highly effective models in five domains. • The paper classifies the models based on respective tasks using a proposed taxonomy. • The characteristics of the surveyed models are deeply explored and analyzed. • Future directions and challenges for transformer-based models are deciphered. [ABSTRACT FROM AUTHOR]

A deep learning-based intelligent system has been developed for the identification and detection of high-density small target pests with the aim of addressing the limitations observed in previous detection systems and manual sorting. The novel system promises to overcome issues of low detection accuracy, low detection efficiency, and serious leakage and error detection. It is expected to significantly improve the efficiency of pest detection, thereby offering a potential solution to the challenges posed by the presence of these pests in agricultural settings. In this paper, we propose an enhanced YOLOv5s model to tackle challenges in pest detection, such as small targets, high densities, and species diversity. Our method embeds Channel Attention (CA) modules into the algorithm and broadens the shallow feature detection scale of the original FPNet, bolstering small-target detection. Network convergence is accelerated using the Kmeans++ algorithm for prior frame generation and YOLOv5s' intrinsic weights for transfer learning. The Efficient Intersection over Union (EIOU) loss function is adopted to address unbalanced data labels. Furthermore, we develop an algorithm to determine pest infestation levels, transforming detection results into pest grade evaluations. These advancements form the core of our proposed intelligent pest identification and detection system. In our enhanced YOLOv5s model, we assessed the system's performance using a dataset comprising six prevalent pest species. This dataset was specifically curated to emulate real-life detection scenarios, featuring small pest targets with similar appearances and imbalanced data labels. We compared our model with a common one-stage model and noted average improvements of 5.5%, 2%, and 3.95% in accuracy, recall, and detection precision respectively across both sparse and dense scenarios. Importantly, the enhanced model maintained a detection time of roughly 10ms per image and was only 1MB larger than the original model, illustrating a balance between accuracy and efficiency. Furthermore, our algorithm enables the assessment of pest grades based on the input detection results, illustrating the degree of pest infestation. The present paper describes the development of an intelligent system for the automatic identification and detection of pests. Experimental results demonstrate the high efficiency and efficacy of the proposed system in identifying and detecting common pests, even in complex scenarios characterized by high pest density and data label imbalance. Notably, the proposed system has the potential to assist practitioners in the fields of agriculture and forestry to make informed decisions on pest control, thereby improving work efficiency and productivity. • An intelligent system for pest identification development was developed. • An improved YOLOv5 model is proposed. • An algorithm was developed to determine the level of pest infestation. [ABSTRACT FROM AUTHOR]

In this paper the performance of different Machine Learning and Deep Learning approaches is evaluated in problems related to green mobility in big cities. Specifically, the forecasting of bike sharing demand in Madrid and Barcelona (Spain) is approached, for different prediction time-horizons, and also a problem of cable car demand forecasting in Madrid city. An important number of predictive variables are considered, which are grouped into four different sets (categorical/calendrical, persistence-based, meteorological and, as a novelty of the paper, information about analogue past instances), whose relevance is studied for all cases. A feature selection mechanism is also incorporated in order to improve the prediction accuracy of the proposed algorithms. A total of 12 different multivariate regression techniques are implemented, covering from Machine Learning methods to time-series Deep Learning approaches. Excellent results in all the prediction problems approached are reported. Finally, the consequences of obtaining accurate prediction in these three problem of green mobility in big cities are discussed. In addition, it is studied how the results could be exported to other similar cases in more general urban mobility studies. Novelties of the work include: (1) Addressing the forecast problem of passenger flow on a cable car using ML and DL multivariate techniques; (2) using the demand of analogous past instances as an additional feature to solve the demand prediction problems; and (3) the extraction of global conclusions about feature relevance when addressing a demand forecasting problem in green mobility. [ABSTRACT FROM AUTHOR]

Imaging spectroscopy integrates traditional computer vision and spectroscopy into a single system and has gained widespread acceptance as a non-destructive scientific instrument for a wide range of applications. The current state of imaging spectroscopy spans diverse applications including but not limited to air-borne and ground-based computer vision systems. This paper presents the current state of research and industrial applications including precision agriculture, material classification, medical science, forensic science, face recognition and document image analysis, environment monitoring, and remote sensing, which can be aided through imaging spectroscopy. In this regard, we further discuss a comprehensive list of applications of imaging spectroscopy, pre-processing techniques, and spectral image acquisition systems. Likewise, publicly available databases and current software tools for spectral data analysis are also documented in this review. This review paper, therefore, could potentially serve as a reference and roadmap for people looking for literature, databases, applications, and tools to undertake additional research in imaging spectroscopy. • A survey on the current state of imaging spectroscopy. • Pre-processing techniques and data acquisition methods are highlighted. • We provide applications of imaging spectroscopy in various domains. • Publicly available databases and software tools for spectral data are presented. [ABSTRACT FROM AUTHOR]

The detection of underwater fish targets is critical for ecological monitoring and marine biodiversity research. However, underwater fish detection is typically constrained by problems including low image quality and variable underwater surroundings. On behalf of further improving the underwater fish detection accuracy in complex underwater environments, this paper proposes a dual-path (DP) Pyramid Vision Transformer (PVT) feature extraction network named DP-FishNet. The backbone network DP-PVT composed from the PVT Network is made up of two feature extraction paths. The first represents the Vision Transformer path, which extracts global features to enhance the distinction between the foreground and background of underwater images. The second is the convolutional neural network path, which enhances the accuracy of detecting small targets by extracting local features. Additionally, to more effectively utilize the feature information extracted by the network, this paper provides a promising solution to employ the content-aware reassembly of features (Carafe) in the feature pyramid network (FPN). The seesaw loss is utilized as a classification loss to address the problem of unbalanced samples caused by the gap in the number of fish populations. According to the experimental findings, the AP and AP50 of the DP-FishNet are 76.0% and 95.2%, respectively. In comparison to currently available advanced two-stage detection algorithms, the quantity of computation and parameters is reduced by approximately 40%. DP-FishNet strengthens the ability to extract global and local features from underwater images and enhances feature reuse. DP-FishNet can be utilized to detect fish targets in actual and complicated underwater habitats. [ABSTRACT FROM AUTHOR]

In recent years, natural language processing (NLP) has gained new territory beyond its traditional use in text mining applications. This paper shows the effectiveness of NLP techniques in assessing the apparent human personality from his/her video transcript, building a bridge between NLP and computer vision-based reasoning. In this paper, a new deep learning model using attention mechanism and bidirectional LSTM layers for estimating the Big-five personality traits is provided and then tested on ChatLearn video dataset. The robustness of the approach is then tested by generalizing the method to two other datasets (B5 corpus and Mypersonality datasets). Several empirical evaluations taking into account the various inputs of the data processing pipeline have been performed to yield optimal model parameters. The developed model is tested on the APA'2016 competition dataset from Chalearn V2 Challenge Workshop for both Big-five personality trait estimation and interview score estimation. We achieved an average of 89.27% in personality trait recognition rate and 89.16% score in the Job Interview challenge. Similar trends of high accuracy score estimation is held for B5-corpus and MyPersonality datasets as well. The results outperformed several state-of-art approaches, demonstrating our approach's feasibility in extending the computer vision approach of first impression personality to natural language processing. This opens up a new direction in multimedia analysis. • New big five personality trait estimation using deep learning and NLP was developed. • The approach was tested successfully on ChatLearn video dataset. • Generalization is demonstrated using B5 corpus and MyPersonality dataset. [ABSTRACT FROM AUTHOR]

Autonomous vehicles (AV)s have become a trending topic nowadays since they have the potential to solve traffic problems, such as accidents and congestion. Although AV systems have greatly evolved, it still have their limitations. For example, Google reported that their AVs have been involved in several collisions and near misses. While most of these collisions and near misses were caused by third parties, the AVs should be able to predict and avoid them. Events like this show that there is still room for improvement in the AV system. This paper aims to present a review of the state-of-the-art algorithms proposed to enable AV behaviour prediction systems to predict trajectories and intentions for pedestrians and vehicles. This will be achieved by using information from previous literature review papers, recent works, and results obtained using well-known datasets. • An AV behaviour prediction system must have a long (> 3s) prediction horizon. • It is a challenging task to model interaction between agents. • Datasets from onboard sensors should be used to predict road agents' behaviour. • Deep learning techniques have shown better results to predict behaviours. • Algorithms should be evaluated using the same datasets and evaluation metrics. [ABSTRACT FROM AUTHOR]

Intelligent Fault-Tolerant Control (IFTC) refers to the applications of machine learning algorithms for fault diagnosis and design of Fault-Tolerant Control (FTC). The overall goal of the FTC is to accommodate defects in the system components while they are in use and maintain stability with little to no performance reduction. These systems are crucial for mission-critical and safety-related applications where the safety of people is at stake and service continuity is crucial. In this review paper, a systematic study has been done for the development of FTC with machine learning, deep learning, and transfer learning algorithms. The challenges and limitations faced with their possible solutions through machine learning theories for the IFTC model are lined up. This paper guides researchers on the different possible types of machine learning algorithms and their advanced forms like deep learning and transfer learning. The differences among these are highlighted by the challenges and limitations of each. The paper is significant such that most of the important literature references from the Scopus database particularly related to important electrical and mechanical industrial problems have been discussed to guide the researchers who want to apply IFTC for specific industrial problems, being the research gap. Finally, future research directions for the development of IFTC are highlighted. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new deformable convolutional network with attention mechanism to deal with the Chinese character of various dynasties. These ancient Chinese characters are hieroglyph with special spatial structure and gradually evolving into modern Chinese characters. However, because of the intrinsic limitations of traditional convolutional networks in the model, the geometric structure of the convolutional module is fixed. Therefore, we build a new architecture called attention-based deformable networks for this task. Besides, we construct a new dataset called the Chinese characters from Various Dynasties Dataset (CCDD), which includes the evolution of Chinese characters between major dynasties. Deformable convolution based on attention mechanism can obtain those more important offsets, resulting in better performance than the original deformable convolution. In the experimental stage, extensive experiments validate the performance of our approach. Firstly, the effectiveness of this network was verified on two text datasets, Mnist(99.38%) and notMnist(98.22%) both comparing with several modules. Finally, the best performance was achieved on the CCDD dataset(91.50%) compared to other classical deep learning networks. This is the first paper to apply deep learning methods to recognize and classify the characters of major dynasties in China. And archaeological workers need to be able to recognize Chinese characters from various periods in China, and our algorithm can assist them in completing this task to a certain extent. It can also help cultural relic enthusiasts roughly determine the age of cultural relics. [ABSTRACT FROM AUTHOR]

Marine ecosystems and their fish habitats are becoming increasingly important due to their integral role in providing a valuable food source and conservation outcomes. Due to their remote and difficult to access nature, marine environments and fish habitats are often monitored using underwater cameras to record videos and images for understanding fish life and ecology, as well as for preserve the environment. There are currently many permanent underwater camera systems deployed at different places around the globe. In addition, there exists numerous studies that use temporary cameras to survey fish habitats. These cameras generate a massive volume of digital data, which cannot be efficiently analysed by current manual processing methods, which involve a human observer. Deep Learning (DL) is a cutting-edge Artificial Intelligence (AI) technology that has demonstrated unprecedented performance in analysing visual data. Despite its application to a myriad of domains, its use in underwater fish habitat monitoring remains under explored. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring. In addition, we provide a comprehensive survey of key deep learning techniques for fish habitat monitoring including classification, counting, localisation, and segmentation. Furthermore, we survey publicly available underwater fish datasets, and compare various DL techniques in the underwater fish monitoring domains. We also discuss some challenges and opportunities in the emerging field of deep learning for fish habitat processing. This paper is written to serve as a tutorial for marine scientists who would like to grasp a high-level understanding of DL, develop it for their applications by following our step-by-step tutorial, and see how it is evolving to facilitate their research efforts. At the same time, it is suitable for computer scientists who would like to survey state-of-the-art DL-based methodologies for fish habitat monitoring. • Marine environments and fish habitats are widely surveyed using underwater cameras. • Deep learning (DL) offers an effective solution for processing underwater videos. • We write a tutorial on the key concepts and steps of DL for underwater monitoring. • We provide a survey of key DL-based visual fish monitoring methods/datasets. • We shed light on challenges/opportunities of DL for underwater fish monitoring. [ABSTRACT FROM AUTHOR]

In this paper we present SurvLIMEpy, an open-source Python package that implements the SurvLIME algorithm. This method allows to compute local feature importance for machine learning algorithms designed for modelling Survival Analysis data. The presented implementation uses a matrix-wise formulation, which allows to speed up the execution time. Additionally, SurvLIMEpy assists the user with visualisation tools to better understand the result of the algorithm. The package supports a wide variety of survival models, from the Cox Proportional Hazards Model to deep learning models such as DeepHit or DeepSurv. Two types of experiments are presented in this paper. First, by means of simulated data, we study the ability of the algorithm to capture the importance of the features. Second, we use three open source survival datasets together with a set of survival algorithms in order to demonstrate how SurvLIMEpy behaves when applied to different models. • Python package implementing SurvLIME algorithm for computing feature importance. • It supports a wide variety of survival models, including deep learning models. • Fast and efficient implementation due to matrix-wise optimisation problems. • An open-source implementation available on GitHub. • Stable release provided to PyPI. [ABSTRACT FROM AUTHOR]

The provision of accurate travel time information holds paramount significance for optimizing traffic operations and management. However, existing approaches typically address multiple travel time learning tasks independently, neglecting the correlation of tasks and necessitating the construction of separate models for each task. As the scale of the road network expands, this practice becomes increasingly infeasible. To bridge this gap, this paper presents an adaptive deep multi-task learning model to solve multiple travel time estimation tasks collaboratively from a citywide perspective. The novel model seamlessly integrates geographical, environmental, and spatial-correlated features, capturing the complex non-linear characteristics of travel time in the urban road network through a deep network learning feature representations layer by layer from the input data. Through the paradigm of deep multi-task collaborative learning, the knowledge learned from one task serves as an inductive bias for other tasks, thereby facilitating the learning of related tasks. By modeling the uncertainty of travel time with a probabilistic model, the weights of jointly learned tasks are adaptively fine-tuned in a principled way. Additionally, a novel algorithm is devised to fully leverage the information embedded within the data with partial information. The proposed methodology is evaluated through a case study on the citywide road network in Beijing, China. Empirical results of extensive experiments demonstrate the adeptness of the proposed model in effectively harnessing information transferred from parallel tasks, adaptively fine-tuning the weights of joint learning tasks, and proficiently exploiting data characterized by incomplete labels, therefore leading to superior performance compared to competing methods. The average estimation error of the four jointly learned travel time estimation tasks is about 28.48%, yielding an improvement of over 4% compared to single-task learning models. The results verify the effectiveness and robustness of the proposed model for travel time estimation in the urban road network. [ABSTRACT FROM AUTHOR]

Existing facial expression recognition (FER) methods are mainly devoted to learning discriminative features from high-resolution images. However, when applied to low-resolution images, their performance drops rapidly. This paper proposes a unified learning framework (namely SR-FER) by cascading the image super-resolution (SR) task and FER task to alleviate the low-resolution challenge. It effectively feeds back expression-related information from the FER network to the SR network, and returns the quality-enhanced expression images via a SR network. Specifically, a multi-stage attention-aware consistency loss module is introduced to help the SR network achieve discriminative feature restoration guided by attention information. Furthermore, a prediction consistency loss module is also developed to encourage the SR network to restore discriminative features by reducing the difference in prediction information between the restored and original normal-resolution images. Therefore, more accurate results are obtained by performing FER on the restored images. We conduct extensive experiments to demonstrate that the proposed low-resolution FER solution can help SR methods restore features favorable for FER while maintaining acceptable FER performance in various resolution degradation scenarios. The proposed method effectively improves the FER challenge under resolution degradation conditions, which is of good reference value for real-world applications. • Design a joint learning framework to alleviate the low-resolution FER challenge. • Design novel attention-aware consistency and prediction consistency losses. • Extensive experiments demonstrate the effectiveness of the proposed solution. [ABSTRACT FROM AUTHOR]

Intelligent defect detection systems based on deep learning have great potential for industrial applications to ensure product quality. However, the success of such data-driven approaches relies heavily on well-established datasets that require considerable time and labor for fine-grained annotation. Actual industrial datasets are often roughly labeled with many inaccurate annotations, thus seriously hindering the performance of defect detection. In particular, the boundaries of surface defects are not only difficult to label reliably, but also to detect accurately. In this paper, a novel approach termed BTDNet (Boundary-guided Task Decomposition Network) is proposed to tackle the challenge posed by inaccurate annotations. BTDNet decomposes the inaccurate original annotation into two accurate annotations, introducing two reliable auxiliary tasks. These auxiliary tasks are supervised separately using the new annotations, enabling BTDNet to mitigate the negative impact of inaccurate annotations during training and learn robust features. An efficient statistics recalibration module is constructed in BTDNet to customize input features for different tasks, while an adaptive feature summation module is employed to merge the outputs of all tasks. As an essential part of BTDNet, both modules are designed with a strong focus on lightweight. Extensive experiments on three publicly available industrial datasets confirm the effectiveness of our method, showing that BTDNet can consistently achieve performance gains across different baselines and input scales. For example, BTDNet clearly improves the F1-score of DeepLabv3 and GCNet methods on the Magnetic-tile dataset by 5.77% and 5.40%, respectively. [Display omitted] • Unreliable annotations lead to poor defect detection, especially on boundaries. • Decomposing unreliable tasks on boundaries helps to learn robust features. • Simple but effective statistics recalibration module and adaptive summation module. • High focus on lightweight, introducing only negligible parameters and calculations. • Consistent performance improvements across different datasets and baselines. [ABSTRACT FROM AUTHOR]

Stuttering is a prevalent speech disorder affecting millions worldwide. To provide an automatic and objective stuttering assessment tool, Stuttering Event Detection (SED) is under extensive investigation for advanced speech research and applications. Despite significant progress achieved by various machine learning and deep learning models, SED directly from speech signal is still challenging due to stuttering speech's heterogeneous and overlapped nature. This paper presents a novel SED approach using multi-feature fusion and attention mechanisms. The model utilises multiple acoustic features extracted based on different pitch, time-domain, frequency domain, and automatic speech recognition feature to detect stuttering core behaviours more accurately and reliably. In addition, we exploit both spatial and temporal attention mechanisms as well as Bidirectional Long Short-Term Memory (BI-LSTM) modules to learn better representations to improve the SED performance. The experimental evaluation and analysis convincingly demonstrate that our proposed model surpasses the state-of-the-art models on two popular stuttering datasets, with 4% and 3% overall F1 scores, respectively. The superior results indicate the consistency of our proposed method, supported by both multi-feature and attention mechanisms in different stuttering events datasets. • We propose A novel attention-based model to learn stuttering events detection (SED). • Our multi-feature fusion approach can improve SED performance. • Our multi-feature fusion model outperforms state-of-the-art methods. • We introduced an attention map along two separate dimensions for SED. • We demonstrate the effectiveness of CBAM in focusing on the salient features. [ABSTRACT FROM AUTHOR]

For the purpose of managing financial risk and making investment decisions, interval stock price forecasting is essential. Currently, decomposition integration frameworks are widely used in point-valued stock price forecasting studies, mainly focusing on mining internal information. However, point forecasts are difficult to adequately capture price uncertainty and may suffer from loss of volatile information. Therefore, this paper proposes an enhanced interval-valued decomposition integration model for stock price prediction based on comprehensive feature extraction and optimized deep learning. Firstly, the interval variational modal decomposition with feedback mechanism (FIVMD) is proposed to extract internal features and can decompose interval values into interval trend and residual. FIVMD not only solves the interval decomposition challenge, but also helps to improve the internal feature extraction capability. Secondly, while considering the influencing factors more comprehensively, appropriate feature selection and compression techniques can effectively achieve external feature extraction, obtain the best influencing factors, and improve the modeling capability of high-dimensional data. Finally, the final prediction results are obtained by modeling the interval trend and residuals separately through the optimization algorithm and deep learning model to improve the prediction accuracy. The results of the empirical analysis reveal that the proposed interval decomposition integrated model has the smallest of the three evaluation metrics, where the values of interval mean average percentage errors (IMAPE) are 1.8188%, 1.1244%, 1.9001%, and 2.1542% respectively. This shows that the model is significantly more accurate and stable than the other comparative models, and it is a successful model for predicting interval-valued stock prices. [ABSTRACT FROM AUTHOR]

Automatic license plate recognition plays an important role in vehicle management. The performance of current license plate recognition systems is significantly affected by large-angle deflection in unconstrained scenes. For addressing this issue, this paper develops a license plate recognition system via a new image correction scheme and an improved CRNN (Convolutional Recurrent Neural Network). In which, the yolov5l is firstly introduced for license plate detection, and it is trained by transfer learning in order to address insufficient data. Then, a new license plate correction scheme and AFF-Net (Adaptive Fusion Feature Segmentation Network) is proposed and applied, which uses the segmentation result of license plate and its original image area for perspective transformation to improve the correction effect. Finally, the channel attention mechanism is added to the CRNN model for license plate recognition, so that a single grid cell of the feature network can obtain more spatial information. The recognition accuracy of the developed license plate recognition system is 98.8 % on CTPFSD (China Temporary Parking Fee System Data), which is much higher than current mainstream license plate recognition systems. More importantly, the developed license plate recognition system has been deployed in the cloud and applied in outdoor parking toll in practice. [ABSTRACT FROM AUTHOR]

Face Recognition is a widely used authentication method deployed for accessing personal digital devices like mobiles and laptops, digital door locks, or various security applications. However, presentation attacks in form of print, video and mask can circumvent such authentication methods. Several solutions based on texture features are proposed in the state-of-the-art; nevertheless, they are sensitive to varying illumination conditions and are less resilient against 2D and 3D attacks. In this paper, we propose a novel 3-stream compact Xception framework (3sXcsNet) consisting of four input streams RGB, MSRCR, MSRCP, and a depth map to form two specific 3-stream fusion (CRF and CPF) to mitigate the illumination scenarios. These streams provide discriminative features. RGB images have rich texture features; MSRCR and MSRCP images are illumination invariant and preserve better chromaticity applied on a color or intensity channel, respectively. In addition, an auxiliary depth map that provides depth information for each pixel makes it easier to analyze features and recognize whether the input image is from a real person or a spoof medium. The 3-stream CRF and CPF represent the features differently and feed independently to a deep CNN compact Xception with a CBAM network. These feature maps are fused using an average fusion mechanism for classification of real and spoof faces. We utilize both 2D and 3D attack datasets namely, REPLAY-ATTACK, CASIA-FASD, OULU-NPU, and 3DMAD to evaluate the effectiveness of the proposed framework. We attain an EER of 1.27% and 0.19% for CRF fusion, as well as 1.48% and 0.20% for CPF fusion when assessing performance on the CASIA-FASD and REPLAY-ATTACK datasets, respectively. On the OULU-NPU dataset, we obtain the most favorable results for two protocols while employing CRF fusion, resulting in ACER metrics of 0.7% and 3.2±2.7%, respectively. On the other hand, CPF fusion outperformed with an ACER of 1.8±1.0% for one protocol. Our findings on intra-database outperform the current state-of-the-art. We also perform cross-database testing to assess the generalization capability of the framework on 2D and 3D attack datasets. Furthermore, the proposed method outperforms both fusion procedures in terms of error rate HTER for intra-database and cross-database testing on the 3DMAD dataset. • Novel 3-stream compact Xception framework to detect face PAs. • Two specific 3-stream input fusion CRF and CPF provide discerning texture features. • A concise deep learning framework offers refined features for PAs classification. • The model mitigates 2D and 3D attacks and is robust against illumination settings. • We outperform several state-of-the-art methods on publicly available datasets. [ABSTRACT FROM AUTHOR]

Models based on extremely deep convolutional networks and attention mechanisms now have powerful feature extraction abilities, improving text recognition performance in natural scenes. However, the very deep network model built with stacked layers brings massive parameters, which limits the application of the text recognition algorithm on storage-constrained devices. In this paper, we propose a lightweight and effective backbone called the Recursive Residual Transformer Network (RRTrN) for scene text recognition. Specifically, by leveraging recursive learning and a combination of convolutional layers and a transformer unit, RRTrN achieves powerful feature extraction while significantly reducing the number of parameters. This reduction in parameters promotes the deployment of our text recognition algorithm on storage-constrained devices, making it more accessible for practical applications. Furthermore, a recursive distillation strategy is presented to balance the recursive learning inference time and performance, enhancing the practicality and efficiency of RRTrN. Extensive experiments on mainstream benchmarks and popular models verify the generalization of RRTrN and achieve state-of-the-art recognition performance on five datasets. Notably, the classical STR model based on RRTrN can achieve a 3 percentage point increase in recognition accuracy or reduce the number of parameters by 80%. • Propose a lightweight and efficient backbone, RRTrN, for scene text recognition. • Present a simple and effective strategy, Recursive Distillation, for computational efficiency issues. • Good generalization on several classical and advanced models. [ABSTRACT FROM AUTHOR]

In 2016, Google proposed Federated Learning (FL) as a novel paradigm to train Machine Learning (ML) models across the participants of a federation while preserving data privacy. Since its birth, Centralized FL (CFL) has been the most used approach, where a central entity aggregates participants' models to create a global one. However, CFL presents limitations such as communication bottlenecks, single point of failure, and reliance on a central server. Decentralized Federated Learning (DFL) addresses these issues by enabling decentralized model aggregation and minimizing dependency on a central entity. Despite these advances, current platforms training DFL models struggle with key issues such as managing heterogeneous federation network topologies, adapting the FL process to virtualized or physical deployments, and using a limited number of metrics to evaluate different federation scenarios for efficient implementation. To overcome these challenges, this paper presents Fedstellar, a novel platform designed to train FL models in a decentralized, semi-decentralized, and centralized fashion across diverse federations of physical or virtualized devices. Fedstellar allows users to create federations by customizing parameters like the number and type of devices training FL models, the network topology connecting them, the machine and deep learning algorithms, or the datasets of each participant, among others. Additionally, it offers real-time monitoring of model and network performance. The Fedstellar implementation encompasses a web application with an interactive graphical interface, a controller for deploying federations of nodes using physical or virtual devices, and a core deployed on each device, which provides the logic needed to train, aggregate, and communicate in the network. The effectiveness of the platform has been demonstrated in two scenarios: a physical deployment involving single-board devices such as Raspberry Pis for detecting cyberattacks and a virtualized deployment comparing various FL approaches in a controlled environment using MNIST and CIFAR-10 datasets. In both scenarios, Fedstellar demonstrated consistent performance and adaptability, achieving F 1 s c o r e s of 91%, 98%, and 91.2% using DFL for detecting cyberattacks and classifying MNIST and CIFAR-10, respectively, reducing training time by 32% compared to centralized approaches. • Open-source platform for training FL models in a decentralized fashion. • Support for diverse FL architectures, network topologies, datasets, and ML/DL models. • User-friendly setup and efficient monitoring of FL processes. • Successful deployment across diverse physical and virtualized devices. • Promising performance in tasks like cyberattack detection and image classification. [ABSTRACT FROM AUTHOR]

Detecting bearing faults helps ensure the healthy operation of machinery and prevents serious accidents. However, fault diagnosis method based on deep learning relies on the correlation between the extracted vibration signal features. It does not consider the causal relationship between fault, noise, and vibration waveform changes, resulting in lower bearing fault diagnosis accuracy under realistic working conditions. This study proposes a graph neural network (GNN) method based on the Granger causality test for bearing fault detection called GCT-GNN to address this issue. The proposed method first performs feature transformation on the original signals to extract time-domain and frequency-domain features, forming a feature matrix. Then, spectral analysis is conducted on both faulty signals with noise and healthy signals to calculate the lag order between them. Subsequently, the Granger causality test is employed to quantify the impact of faults and noise on signal changes, and the quantified results are used to calculate weights, constructing an adjacency matrix. The completed adjacency matrix and feature matrix are input into the GNN for feature mapping, last classifying the bearing fault data. In this paper, five models, Deep residual shrinkage Network (DRSN), GNN, Support Vector Machines(SVM), Random Forests (RF), and Convolutional Neural Network(CNN) were selected, and comparative experiments were carried out on two public data sets. The results show that, compared with other models, GCT-GNN has better anti-noise ability and is better than other models in various fault diagnoses. • The effect of noise on vibration signal is quantified by Granger causality test. • The lag order was calculated by using spectrum analysis method. • The method of calculating weight by causality is proposed to construct causality graph. • Construct a graph neural network using causal graphs. [ABSTRACT FROM AUTHOR]

Deep learning has emerged as a powerful tool in various domains, revolutionising machine learning research. However, one persistent challenge is the scarcity of labelled training data, which hampers the performance and generalisation of deep learning models. To address this limitation, researchers have developed innovative methods to overcome data scarcity and enhance deep model learning capabilities. Two prevalent techniques that have gained significant attention are transfer learning and self-supervised learning. Transfer learning leverages knowledge learned from pre-training on a large-scale dataset, such as ImageNet, and applies it to a target task with limited labelled data. This approach allows models to benefit from the learned representations and effectively transfer knowledge to new tasks, resulting in improved learning performance and generalisation. On the other hand, self-supervised learning focuses on training models using pretext tasks that do not require manual annotation, allowing them to learn valuable representations from large amounts of unlabelled data. These learned representations can then be fine-tuned for downstream tasks, mitigating the need for extensive labelled data. In recent years, transfer and self-supervised learning have found applications in various fields, including medical image processing, video recognition, and natural language processing. These approaches have demonstrated remarkable achievements, enabling breakthroughs in areas such as disease diagnosis, object recognition, and language understanding. However, while these methods offer numerous advantages, they also have limitations. For example, transfer learning may face domain mismatch issues between the pre-training and target domains, while self-supervised learning requires careful design of pretext tasks to ensure meaningful representations. This review paper explores the recent applications of these pre-training methods in various fields within the past three years. It delves into the advantages and limitations of each approach, assesses the performance of models employing these techniques, and identifies potential directions for future research. By providing a comprehensive review of current pre-training methods, this article offers guidance for selecting the best technique for specific deep learning applications to address the data scarcity issue. • Differences between Transfer Learning and Self-Supervised Learning. • Applications of Transfer Learning and Self-Supervised Learning. • Advantages and Limitations of Transfer Learning and Self-Supervised Learning. • Recommendations for the Use of Each Approach in Specific Applications. [ABSTRACT FROM AUTHOR]

Endoscopic ultrasonography (EUS) is a valuable imaging modality for diagnosing gastrointestinal submucosal tumors (SMTs). However, inherent content variations in EUS images due to gastrointestinal tract mobility and handheld ultrasound instability challenge the saliency of SMTs' visual features. The presence of fine-grained inter-class and large intra-class differences further complicates EUS-based diagnosis. To address these issues, this paper presents a novel Multi-Attribute Guided Contextual Attention Network (MAG-CA-Net) for interpretable SMT recognition in EUS. Inspired by endoscopists' clinical diagnosis expertise, our framework initially localizes abnormal areas based on echo attributes and subsequently determines tumor categories using contextual semantics. Experimental results demonstrate the effectiveness of MAG-CA-Net, exhibiting improved recognition recall and precision rates for gastrointestinal stromal tumor, leiomyoma, and pancreatic rest. Specifically, the MAG network facilitates abnormal area localization, while the CA mechanism guides the model to focus on the most discriminative tumor-context-associated regions. The proposed method achieved an average classification accuracy of 93.16%, an average precision of 93.17%, a weighted recall of 93.16%, and an average F1-score of 93.15 % for the three disease categories. The proposed approach provides crucial guidelines for data collection standards and model development in the clinical diagnosis process of SMTs under EUS. Its interpretability analysis enhances the credibility of clinical physicians towards assisted diagnostic methods based on deep learning. The source code will be publicly available at https://gitee.com/HangbinZheng/mag-ca-net. • Hierarchical identification strategy combining clinical knowledge and data-driven models for submucosal tumor detection. • Contextual attention mechanism enhances tumor recognition using gastrointestinal layered features. • Improved adaptability to endoscopic ultrasound image variation and fine-grained feature extraction compared to state-of-the-art models. • Visual interpretability close to experienced endoscopists enhances method's interpretability. [ABSTRACT FROM AUTHOR]

This paper proposes a new model for creating fuzzy time series and conducting future forecasts using the Bi-directional Long Short-term Memory model (Bi-LSTM). This study contributes by using both classical machine learning and deep learning techniques. Firstly, a self-updating clustering algorithm is used to determine the optimal number of clusters for time series. Secondly, the study describes a method for determining the fuzzy relationship between elements of time series and the identified clusters. Thirdly, the developed model creates the fuzzy time series based on the established rules. Finally, forecasting for the future value using the Bi-LSTM model with some improvement in optimizing the parameters and increasing the accuracy of forecasting result. The steps of the proposed model are clearly illustrated by a numerical example. Moreover, this model also tested on various datasets, including M3, M4, and M5 datasets. By evaluating its performance using parameters such as sMAPE and RMSSE, the proposed model has shown significant enhancements when compared to existing models. Additionally, the model's effectiveness is further demonstrated through its successful application in forecasting the VN index stock in Vietnam. [ABSTRACT FROM AUTHOR]

Remote sensing scene understanding is crucial for extracting valuable information from high-resolution images, including object detection and classification. Traditional object detection methods face challenges in handling the diverse scales, orientations, and complex backgrounds present in remote sensing data. In this paper, we propose a novel remote sensing scene understanding system called multiscale-attention R-CNN (MSA R-CNN), which incorporates a super multiscale feature extraction network (SMENet) for enhanced feature extraction from multiscale images, an adaptive dynamic inner lateral (ADIL) connection module to tackle information loss in feature pyramid networks (FPN), and a distributed lightweight attention module (DLAM) to refine feature information processing. Furthermore, a new dataset combining the DIOR and DOTA datasets is introduced to extract the background information of detected objects and evaluate the proposed system's performance. MSA R-CNN achieved an mAP of 74.37% on the DIOR dataset when the gamma value was set to 0.2 and 81.97% on the DOTA dataset when the gamma value was set to 0.1 with the same learning rate, outperforming state-of-the-art models on both datasets. The proposed system demonstrates significant improvements in both object detection and background information extraction, providing a comprehensive solution for remote sensing scene understanding. • MSA R-CNN is proposed for remote sensing scene understanding. • SMENet enhances multi-scale feature extraction in MSA R-CNN. • ADIL module tackles FPN information loss. • DLAM refines feature information processing in FPN. • Achieved state-of-the-art performance on DIOR & DOTA dataset. [ABSTRACT FROM AUTHOR]

Retrieving similar images is becoming an urgent need for us with the continuous growth of large-scale data. However, whether the dominant image retrieval methods are Convolutional Neural Networks (CNNs) or the recently emerging Visual Transformer (ViT), their complex computation, insufficient feature extraction, and mismatched weights greatly influence the efficiency and retrieval accuracy. In this paper, we propose a Pooling-based Visual Transformer with low complexity attention hashing (PTLCH) for image retrieval. First, a backbone network for Pooling-based Vision Transformer (PiT) feature learning is designed to combine the pooling in CNN and the ViT to achieve the purpose of spatial dimensionality reduction while learning rich semantic information. Second, a low complexity attention (LCA) module is incorporated into PiT, which works by combining the positional deviation with the key matrix and the value matrix and then matrix multiplying with the query matrix. LCA explores rich contextual information to enable network learning of more granular feature information. Finally, a new loss framework is proposed where we focus on the effect of difficult and erroneous samples on accuracy. By using different improved cross-entropy losses, better weights are assigned to the learning samples of our network, which effectively improves learning hash coding. We have conducted extensive experiments on three public datasets, CIFAR-10, ImageNet100, and MS-COCO, which have the highest mean average precision of 93.76%, 92.62%, and 90.60%, respectively. [Display omitted] • An end-to-end learning framework for image retrieval is proposed, namely PTLCH. • LCA modules are embedded on top of the network. • A new deep hash learning loss framework is designed. • The superior performance of PTLCH method is demonstrated experimentally. [ABSTRACT FROM AUTHOR]

The presence of hard exudate (EX) lesions is an early clinical symptom of Diabetic Retinopathy (DR); its accurate segmentation is essential for diagnosis and treatment. Automatic segmentation of EX lesions is challenging because they have a wide range of sizes, shapes, and brightness, and can be confused with soft exudate lesions. In this paper, we present and assess an efficient segmentation method for interactively segmenting EX lesions in fundus images from DR patients. Our approach consisted of two main stages: (1) the first stage generates an automatic segmentation using a CNN-based method. (2) If the obtained segmentation is suboptimal, the second stage is used to refine each inaccurately segmented region using a CNN-based interactive segmentation method. To train and evaluate method performance, we used two public datasets (IDRiD and DDR) and two local datasets from two medical centers (EHos and EWT) containing a wide range of EX lesions. Several state-of-the art CNN-based models and the proposed method were trained and evaluated on IDRiD, DDR and EHos datasets using the area under the precision–recall curve (AUPR), the dice similarity coefficient (Dice) and the intersection over union (IoU). In addition, EWT dataset was used to evaluate the methods without retraining the CNN models. The evaluation results using IDRiD dataset showed that the proposed method achieved AUPR , mean Dice and mean IoU scores of 0.893, 76.6% and 62.4%, respectively, which are comparable to the state-of-the-art method on EX segmentation. When evaluated on the DDR dataset with large portion of small EX lesions, the proposed method achieved AUPR , mean Dice and mean IoU scores of 0.693, 60.8% and 45.5%, yielding state-of-the-art performance. Furthermore, with the main evaluation metric being AUPR , the evaluation results on two datasets, EHos and ETW, with scores of 0.689 and 0.667 demonstrated superiority of the proposed method over the other segmentation methods. The proposed two-stage CNN-based segmentation method is an useful approach for accurately segmenting EX lesions in color fundus images, enabling lesion quantification to aid experts in treating patients with DR. The source code, the trained CNN models and the data are publicly available at: https://github.com/dvquang2000/Interactive_HardExudates_Segmentation. • A two-stage CNN-based method was proposed for accurate EX lesions segmentation in fundus image • Multiple datasets in several hospitals and countries were included in the study • An assessment of the proposed method on two different field of view in image acquisition was performed • State-of-the-art and the proposed methods were tested with and without retraining the CNN models • The proposed method outperformed state-of-the-art methods in most evaluation scores [ABSTRACT FROM AUTHOR]

For achieving better performance, the majority of deep convolutional neural networks have endeavored to increase the model capacity by adding more convolutional layers or increasing the size of the filters. Consequently, the computational cost increases proportionally with the model capacity. This problem can be alleviated by dynamic convolution. In the case of pixel-level labeling, existing pixel-level dynamic convolution methods have a smaller scanning area than ordinary convolution or image-level dynamic convolution and are thus unable to exploit fine contextual information. As a consequence, pixel-level dynamic convolution is more sensitive to large-scale varying objects and confusion categories. In this paper, we propose a scale-pyramid dynamic atrous convolution (SDAConv) and exploit multi-scale pixel-level features in finer granularity, in order to efficiently increase model capacity, exploring contextual information, capture detail information and alleviate large-scale variation problem at the same time. Through kernel engineering (instead of network engineering), SDAConv dynamically arranges atrous filters in the individual convolutional kernels over different semantic areas at dense scales in the spatial dimension. By simply replacing the regular convolution with SDAConv in SOTA architectures, extensive experiments on three public datasets, Cityscapes, PASCAL VOC 2012 and ADE20K benchmarks demonstrate the superior performance of SDAConv on pixel-level labeling tasks. • A new scale-pyramid dynamic convolution(SDAConv) is proposed. • SDAConv is truly dynamic and multiple receptive fields. • SDAConv can improve localization accuracy and explore rich contextual information. • SDAConv can be used in a plug-and-play fashion in any architecture. [ABSTRACT FROM AUTHOR]

• The improved YCbCr models are established for initial fire segmentation under reflection and non-reflection conditions. • We propose entropy function of the average contrast and uniformity to determine the optimal threshold of region growth. • Judgment index is proposed by the variation coefficient of area, dispersion of centroid and circularity. Fire is one of the mutable hazards that damage properties and destroy forests. However, fire-like objects easily influence many existing image-based fire detection methods. In order to improve the performance of fire identification, this paper proposes a new fire identification algorithm by merging fire segmentation and multi feature fusion of fire. First, the improved YCbCr models in the reflection and non-reflection environment are constructed according to the color model. Simultaneously, the reflection and non-reflection conditions can be judged according to the segmented area. Second, the seed points are determined according to the weighted average of centroid of each connected region. Simultaneously, the fine segmentation of fire image is implemented according to the entropy of average contrast and uniformity within the connected region. Experiments show that the segmentation is not affected by image noises. Finally, the quantitative indicators of fire identification are given according to the coefficient of variation of area, the dispersion of centroid and the circularity. Cases show that the proposed identification method of fire not only accurately identifies fire, but also has a lower computation complexity than the deep learning method. [ABSTRACT FROM AUTHOR]

In recent years, the enormous development of Machine Learning, especially Deep Learning, has led to the widespread adoption of Artificial Intelligence (AI) systems in a large variety of contexts. Many of these systems provide excellent results but act as black-boxes. This can be accepted in various contexts, but there are others (e.g., medical ones) where a result returned by a system cannot be accepted without an explanation on how it was obtained. Explainable AI (XAI) is an area of AI well suited to explain the behavior of AI systems that act as black-boxes. In this paper, we propose a model-agnostic XAI framework to explain the behavior of classifiers. Our framework is based on network theory; thus, it is able to make use of the enormous amount of results that researchers in this area have discovered over time. Being network-based, our framework is completely different from the other model-agnostic XAI approaches. Furthermore, it is parameter-free and is able to handle heterogeneous features that may not even be independent of each other. Finally, it introduces the notion of dyscrasia that allows us to detect not only which features are important in a particular task but also how they interact with each other. • Performing model-agnostic Explainable AI on classifiers. • Introducing the concept of dyscrasia of feature occurrences. • Assessing the relevance of a feature during classification. [ABSTRACT FROM AUTHOR]

Smoke detection is essential for disaster management and timely response to fires. However, popular deep learning-based algorithms ignore key traditional features such as color and texture, causing limitations in detection. Therefore, a lightweight edge-guided smoke detection network (ESmokeNet) is proposed in this paper. First, edge cues are refined from the low-level maps and integrated into the network features according to the color channels. Then, to increase the smoke feature extraction capability, we design a mutual context embedding module (MCE). It enhances multiscale context and multilevel semantic information based on attention and chiral features. Finally, the classification and localization tasks are decoupled to pursue the usability of network features directed by smoke edges, and an EdgeDet head is proposed. Extensive experimental and visualization results have confirmed that ESmokeNet can successfully capture smoke edges and that it has significant superiority over the existing state-of-the-art methods and specialized smoke detection algorithms. In addition, the test data show that our ESmokeNet is only 5.42 M and is a lightweight network suitable for smoke detection tasks. Code is available at https://github.com/jingjing-maker/ESmokeNet. • Propose a lightweight edge-guided smoke detection network, termed ESmokeNet. • Feature extraction is enhanced by a mutual context embedding module (MCE). • An anchor-free EdgeDet head pursues an effective fusion of edge cues and network. • ESmokeNet is only 5.42M and can be embedded on resource-constrained devices. [ABSTRACT FROM AUTHOR]

Unsupervised domain adaptation (UDA) is used to extend the model working on well-annotated source data to unlabeled target data. However, in practice, due to privacy and storage issues, we can only obtain the well-trained source model. In this paper, we focus on this scenario named source-free domain adaptation (SFDA). At present, nearest neighbors-based SFDA methods assume that the target features extracted by the source model can form clear clusters, and align samples with their neighbors. However, due to the domain discrepancy, adjacent features may belong to different categories. We propose consistency regularization-based mutual alignment (CRMA) to address this problem. Firstly, we randomly augment each target sample. Due to the domain discrepancy, it may lead to negative transfer if we align them directly. Therefore, secondly, we leverage the information maximization loss to all target and augmented samples, improving the performance of mutual alignment. Finally, we mutually align original samples and augmented samples. It improves the ability of the model and increases the variety of samples to alleviate the phenomenon that incorrectly aligning samples when aligning with neighbors. CRMA achieves state-of-the-art performance on 3 popular cross-domain benchmarks. Compared with the original method, CRMA has improvements of 0.4% up to 89.4%, 1.9% up to 72.2%, and 1.9% up to 85.9% on 3 datasets respectively. At the last, we verify the effectiveness of each part of CRMA through ablation experiments and use a series of experiments to analyze CRMA in detail. • We propose the mutual alignment strategy. • We design the mutual alignment loss based on entropy. • We leverage information maximization and entropy to enhance predictions. • We conduct a series of experiments to demonstrate the effectiveness of CRMA. • Our method outperforms the SOTA methods on three popular benchmarks. [ABSTRACT FROM AUTHOR]

Automatic modulation recognition (AMR) has garnered significant attention in both civilian and military domains, with applications ranging from spectrum sensing and cognitive radio (CR) to the deterrence of adversary communication. Index modulation (IM) represents an innovative digital modulation technique that exploits the indices of parameters of communication systems to transmit extra information bits. This paper aims to examine the performance of a convolutional neural network (CNN)-based AMR across various IM systems, including spatial modulation (SM), quadrature spatial modulation (QSM), and generalized spatial modulation (GSM) with eight digital modulation schemes. In this study, we leverage confusion matrices, receiver operating characteristic (ROC) curves, and F1 scores to illustrate the recognition model's outputs. • Modulation recognition models exploit deep learning algorithms effectively. • Index modulation is a novel technique for conveying additional information. • Three datasets are created for the recognition of eight digital modulation schemes. • Three modulation recognition scenarios analyze the system's performance. • Quadrature amplitude modulation types are more robust than phase shift keying types. [ABSTRACT FROM AUTHOR]

The financial market, as a complex nonlinear dynamic system frequently influenced by various factors, such as international investment capital, is very challenging to build trading strategies from the obtained market information. Deep Reinforcement Learning (DRL) combines the perceptual capability of deep learning and the control decision making capability of reinforcement learning to learn the mapping between financial market states and trading decisions by interacting with the environment. In this paper, an enhanced stock trading strategy, denominated efficient deep State-Action-Reward-State-Action (SARSA), is presented to tackle the algorithmic trading problem of determining optimal trading positions in the daily trading activities of the stock market. This algorithm is recognized for its properties of stable learning and convergence, attributes of critical significance within the financial domain, where stability assumes paramount importance, and excessive risk-taking should be averted. Furthermore, a novel deep network architecture called Bidirectional Long Short-Term Memory (BiLSTM)-Attention is introduced to address the challenge of accurately presenting the complex and volatile stock market. The BiLSTM-Attention greatly enhances the network's capacity to recognize key features and patterns in stock market data, allowing agents to focus on the most relevant aspects of the data. Evaluations on DJI, SP500, GE, IXIC datasets from January 1, 2008 to December 31, 2022 show that our efficient deep SARSA algorithm outperforms a wide range of traditional strategies (B&H, S&H, MR, TF) and DRL-based strategies (TDQN, DQN-Vanilla). For example, on the IXIC dataset, the efficient deep SARSA strategy achieves an attractive Cumulative Return (CR) of 582.17% and a Sharpe Ratio (ShR) of 1.86, outperforming all other methods. These experimental results prove the performance of our method in enhancing stock trading strategies. • Proposed the efficient deep SARSA model for algorithmic trading. • Proposed a network BiLSTM-Attention to extract key features in stock data. • The proposed efficient deep SARSA outperforms other baseline methods (TDQN etc.). [ABSTRACT FROM AUTHOR]

How to identify the screen-shot image is an important branch of image source forensics. Although the physical feature of moiré patterns may be left after LCD recapture operation, this feature may also be well concealed with the development of deep learning-based techniques for demoiréing operation. In this paper, to solve the forensic problem of screen-shot and demoiréd image operation, we select several typical demoiréing algorithms to generate datasets for training and detecting demoiréd images. Meanwhile, we also perform the task of identifying specific demoiréing algorithms. DenseNet can extract fine demoiréd image features because of its enhanced feature reuse and improved model performance. The re-attention mechanism is also added to our designed network to extract the image's global features and avoid the self-attention collapse problem in ViT. Therefore, we design a network based on DenseNet cascaded with DeepViT structure to detect the demoiréd image and identify its corresponding demoiréing algorithm. In a large number of comparison experiments of various typical image recapture detection methods, our method achieves the best detection results and network performance. • Improve the generality of the screen-shot image detection task. • Propose a network dedicated to the demoiréing algorithm identification. • Analyze the features of DenseNet and DeepViT in demoiréd image forensics. • Has a detection accuracy of 0.9126 in the generated demoiréd image dataset. [Display omitted] [ABSTRACT FROM AUTHOR]

Deep learning-based automatic sleep staging methods have been widely applied for sleep scoring and sleep diagnosis. However, most methods consider only a single temporal scale when dealing with sleep signals. Furthermore, these methods are limited to target only a single-age group or single dataset. In this paper, we propose a multi-scale temporally focused sleep staging model, MAGSleepNet, which can be used for multi-age groups simultaneously. MAGSleepNet consists of (1) a group age classification (GAC) module that can offer a preliminary screening on epidemic estimation of multiple age, (2) a dimensional expansion module (DEM) that can expand the dimension of the input signals, (3) a sequential multi-scale convolutional neural network (SMCNN) that extracts multi-scale features and short-time temporal information, and (4) sequence temporal encoder (STE) that extracts sequential temporal information. In addition, two auxiliary tasks are used to complement the short-time temporal information and to reassign the probabilities of different age groups to enhance the model robustness, respectively. The MAGSleepNet is evaluated in adult, child and infant, tested on MASS, CHAT, and CHFU datasets with accruacy of 86.7%, 80.1% and 66.5%, outperforming the state-of-the-art methods. Based on the excellent performance of the proposed method, it is expected to pave the way for automatic sleep staging methods with strong generalizability for multiple age groups. • A novel model is proposed for simultaneous sleep staging in multi-age groups. • The hierarchical architecture of model ensures the effective feature extraction. • An auxiliary task is used to provide temporal information on different scales. • Numerous experiments are conducted to evaluate the proposed model. [ABSTRACT FROM AUTHOR]

In the construction of intelligent transportation systems, short-term traffic flow prediction is of great significance for the advancement of traffic network management. But due to the presence of many complex factors in both spatial and temporal domains, it remains a complex and challenging task. Existing literature usually employs the convolutional neural network (CNN)-based methods in capturing spatiotemporal correlations. These CNN-based methods often use a single-channel data structure to represent different periodic patterns, which makes the model susceptible to over-parameterization when capturing periodic dependencies and prone to information loss after convolution. To overcome these limitations, this paper presents a hybrid deep learning method for short-term traffic flow prediction. In this method, a video-shaped multi-channel data structure is designed to represent different periodic patterns more efficiently. Next, a new mixed-pointwise convolution is introduced for capturing periodic dependencies without the negative impacts mentioned above. Lastly, an improved channel attention mechanism is proposed to learn channel inter-dependencies with controllable parameter usage. The proposed method is lightweight, yet highly effective. Compared to the state-of-the-art baseline method, it reduces the root mean squared error by up to 6.7% on the PeMSD4 dataset and 13.3% on the PeMSD7 dataset, while also achieving substantial improvement in two additional metrics, exhibiting strong robustness and great scalability across various settings. [ABSTRACT FROM AUTHOR]

Vector fonts, serving as the fundamental format of fonts, play a significant role in modern media. Described by a set of mathematical equations, vector fonts enable style modifications by adjusting drawing parameters, making them favored by font artists and designers. Due to the non-structural nature of vector font data, the task of vector font generation resembles sequence generation. Existing methods, limited in handling long sequences, are only capable of synthesizing simple character vector fonts. In this paper, we propose a dual-modal learning strategy to convert raster glyph images to vector glyph images in an end-to-end manner. Specifically, by employing vector quantization, we comprehensively utilize the dual-modal information of vector fonts. We quantize image and sequence modal features using a shared codebook, mapping them to the same discrete space and aligning them. Through aligned features, we reconstruct raster glyph images and vector glyph images. During the transformation process of vector glyph data, we redesign the Transformer module for vector glyph data, leveraging multiple stacked sliding window attention mechanisms to model local and global information. By integrating reversible residuals with attention and feedforward layers within the Transformer module, we enhance the model's capability and stability in handling long sequence data without sacrificing accuracy. Finally, we perform cross-modal model distillation to obtain the model's backbone network. We further refine the backbone network using a differentiable rasterizer to minimize error accumulation during sequence generation. Qualitative and quantitative results demonstrate that our method achieves high-quality synthesis results in complex Chinese character glyphs. The synthesized vector fonts can be easily converted into TrueType fonts for practical use, holding valuable applications for anyone interested in personalized vector font styles. • A bimodal learning strategy is proposed to generate vector images from glyph images. • Alignment of word image modalities and sequence modalities mapped to discrete space. • The ideas of Sliding Window attention and RevNet are used in Transformer. • Vector images are generated from raster images by cross-modal model distillation. • Complex vector font synthesis is achieved with important application value. [ABSTRACT FROM AUTHOR]

This paper presents a novel end-to-end oversampling-classification approach, which we refer to as imbalanced data-classifying generative adversarial network (ImbGAN), for imbalanced data classification. ImbGAN has a classifier-embedded structure within a GAN and consists of five components: (1) generator, (2) discriminator, (3) classifier, (4) storage for misclassified minority class data, and (5) storage for artificial minority class data. By iterative interaction with the embedded classifier, the first two components generate artificial minority class instances that are similar to minority class instances misclassified by the classifier. Therefore, these three networks are iteratively and simultaneously trained. The misclassified and artificial minority class instances are stored in the fourth and fifth components, respectively. These two components are also updated as iterations proceed. Our method obtains the final classification model from a single learning process, while most artificial data generation methods for imbalanced data classification go through an additional process for training classifiers after artificial data generation. Numerical experiments based on tabular, image, and text datasets confirm that the proposed method outperforms well-known synthetic sampling methods. [ABSTRACT FROM AUTHOR]

Financial fraud is a persistent problem in the finance industry that may have severe consequences for individuals, businesses, and economies. Graph Neural Networks (GNNs) are a class of deep learning models designed to operate on graph data structures that consist of nodes and edges connecting them. GNNs have emerged as a powerful tool for detecting fraudulent activities in complex financial systems because they can analyze the network structure of financial transactions, capturing patterns and anomalies that traditional rule-based and machine learning methods might miss. The objective of this systematic review is to provide a comprehensive overview of the current state-of-the-art technologies in using Graph Neural Networks (GNNs) for financial fraud detection, identify the gaps and limitations in the existing research, and suggest potential directions for future research. We searched five academic databases, including Web of Science, Scopus, IEEE Xplore, ACM, and science direct using specific keywords and search strings related to graph neural networks, financial areas, and anomaly detection to identify relevant publications, resulting in a total of 388 unique articles. We selected the relevant publications based on the inclusion, exclusion, and quality assessment criteria, and 33 articles were included in the review. In addition, forward snowballing was used to identify relevant papers that were not captured in the initial search. Data was extracted from the selected articles, then analyzed and summarized to identify current state, gaps, and trends in the literature. Our review presents a new taxonomy of GNNs applied in financial fraud detection and identifies potential research directions in this field. We find that GNNs applied to financial fraud detection have mostly been employed in a supervised or semi-supervised manner, with limited exploration of unsupervised approaches. In addition to financial areas, we explore the different types of graphs such as homogeneous, heterogenous, static, temporal, and dynamic graphs, and investigate the various learning mechanisms and anomaly types studied. We also note a lack of research on edge-level and graph-level anomaly detection commonly employed in financial domain. [ABSTRACT FROM AUTHOR]

Accurate photovoltaic (PV) power forecast is crucial for carbon neutrality. Current researches on PV power forecast mainly focus on using temporal information from single PV station, and the spatial information in multiple PV power stations are often neglected. To address this problem, this paper introduces Moran index to verify the spatial autocorrelation of PV power for the first time, uses Granger causality test and transfer entropy to reveal the spatial information gain in PV power forecast for the first time, and proposes a novel spatial–temporal probabilistic PV forecast method using deep Graph Gated Recurrent Unit (GraphGRU) network-based spatial–temporal multi-task learning and Kernel Density Estimation (KDE). Deep GraphGRU combines the advantages of Graph Convolutional Network (GCN) in spatial feature extraction and the advantages of Gated Recurrent Unit (GRU) network in temporal feature extraction, and thus has strong ability to extract spatial–temporal information in historical data of multiple different PV power stations. Through GraphGRU, temporal dependency information extracted from historical data of multiple PV stations can promote each other to improve the forecast accuracy of each PV stations. KDE is used for estimating the joint probabilistic density function and giving the spatial–temporal probabilistic confidence interval of PV power. Experiments were performed in the five-year actual PV power data from 11 provinces of Belgium and the three-year solar irradiation data from 12 places in China to verify the superiorities of the proposed method. Comparison with conventional spatial–temporal and temporal forecast methods show that the proposed GraphGRU-based spatial–temporal forecast method can extract the spatial–temporal information from multiple PV power stations well and significantly outperform conventional temporal forecast methods. [ABSTRACT FROM AUTHOR]

The enhanced answer evaluation system is a cutting-edge automated tool that evaluates subjective answers in various contexts, such as educational assessments, surveys, and feedback forms. The proposed system leverages Natural Language Processing (NLP) and deep learning techniques to analyse subjective answers and provide evaluation scores with precision. Students' answers are evaluated based on various criteria, such as keywords, context, relevance, coherence, and similarity. This paper introduces an architecture for a subjective answer evaluator using three main aspects: detection of keywords, similarity matrix, and presence of named entities. It combines the three aspects and provides a final score. It provides a standardized mechanism to score a given user answer compared to the particular model answer without human prejudice. This research aims to transcend traditional methodologies that predominantly utilize keyword or keyphrase scoring (text-based similarity) to determine the final score of an answer without delving into its technical intricacies. The semantic similarity (vector-based) employs vector data representations for score calculation. This approach necessitates partitioning data into multiple vectors for a comprehensive analysis. While text similarity is effective for short answers, its efficacy diminishes as the length of the answer increases. Therefore, this study emphasizes the critical role of similarity scoring and Named Entity Recognition (NER) scoring in evaluating more extended responses based on the stsb-en-main dataset (short answers) and a custom dataset with 190 records. This research reveals its remarkable performance, which excels through a dynamic three-pronged approach: keyword scoring, semantic similarity, and NER scoring with models like Yet Another Keyword Extractor (YAKE), SimCSE and Camembert. These three independent components synergize to produce unmatched results, establishing a new standard in the field. This enhancement led to Root Mean Square Error (RMSE) scores of 0.031 (optimized error rate) and an impressive 71%+ accuracy for our comprehensive system. This achievement surpasses existing works, which typically reached accuracies ranging between 40%–60% for long answers. • Evaluation using a three-pronged approach of Keyword, Semantic & NER scoring. • Semantic scores enhance similarity matrix grading and impact evaluation. • NER scores address proper noun issues. • Deep Neural Network predicts actual marks effectively. • The best-performing LLM models are YAKE, SimCSE and Camembert. [ABSTRACT FROM AUTHOR]

Understanding the attitudes of users about different topics through their huge amount of comments and opinions on social networks is an emerging hot issue. Analyzing the sentiments of these posts and comments provides useful information utilized for many applications. Multimodal sentiments often contain more information than single-modal sentiments, including text, image, audio and video, which leads to better performance of multimodal sentiment analysis (MSA) compared to single-modal sentiment analysis (SSA). In this paper, a capsule network-based deep ensemble transfer learning approach, called DSY-ETL-MSA, is proposed for MSA on images and texts, the results of which are fused using Yager theory. Capsule networks and ensemble learning methods improve classification performance, and the deep transfer learning approach reduces the training time. A hybrid deep architecture is used for automatic feature extraction in the proposed method. For analyzing the sentiment of image modality, a pre-trained VGG16 model, fine-tuned on the datasets, is used to extract high-level features for image classification. A capsule convolutional neural network (CNN) is also separately used for extracting image features and classifying them. In the text modality, the pre-trained GloVe model is exploited to embed words and 2 separate classifiers are employed for text classifications. The Yager fusion rules are finally used for early and late fusions of the results of the classifiers. The results of the text and image classifiers are combined separately as the early fusion and the final results of them are fused at the decision level as the late fusion. The proposed model is empirically evaluated on the MVSA and T4SA datasets. The significant performance improvement compared to a variety of former methods is shown in the experimental results. The final accuracies obtained by the proposed method on the MVSA and T4SA datasets are 0.9866 and 0.9996, respectively. [ABSTRACT FROM AUTHOR]

Point cloud classification has emerged as a vital research area in several emerging applications, including robotics and autonomous driving. However, discriminative feature learning has long been a challenging issue owing to the irregularity and disorder of point clouds. Recently, although Transformer-based methods can achieve high accuracy in point cloud learning, plenty of Transformer layers bring huge computation and memory consumption. This paper presents a novel hierarchical local–global framework based on Transformer network for point clouds, named TNPC. TNPC contains two serial Stages implementing downsampling operation, and each Stage is composed of a local feature extracting (LFE) block and a global feature extracting (GFE) block, which can reduce computation and memory consumption obviously. LFE block consists of two parallel branches, namely Transformer branch and shared multilayer perceptron (SMLP) branch, which are designed to learn relevant feature of any two points and local high-dimensional semantic features of each point between sampling centroid and its neighborhoods, respectively. The proposed two parallel branches not only improve the feature extraction effect, but also reduce the computation and memory consumption. GFE block consists of a center point contact (CPC) module and a global point cloud transformer layer (PCTL) module, which can improve the effect of global features extracting without adding the number of parameters and computation. The performance of our method is validated experimentally on ModelNet40 and ScanObjectNN datasets. Our method improves the mean accuracy on each category (m A c c) to 91.6% and 79.8% on the ModelNet40 dataset and ScanObjectNN dataset, respectively. In terms of efficiency, our method leads to a significant reduction, with only 4.73MB parameters and only 1.91GB floating-point operations (FLOPs). Experimental results demonstrate that the proposed method achieves state-of-the-art performance on classification accuracy and efficiency. • A novel hierarchical local–global framework is proposed for 3D point clouds. • LFE block contains parallel Transformer branch and SMLP branch. • LFE block can learn local high-dimensional semantic features of each point. • GFE block is designed to extract the global feature of point cloud effectively. [ABSTRACT FROM AUTHOR]

Predicting human motion is a critical requirement in various applications, with particular significance in the construction sector. This task presents significant challenges due to the diverse nature of human actions and the complexities of converting 2D image coordinates to real-world space. In response to these challenges, this paper introduces an innovative deep learning-driven approach to forecast human motion trajectories, with a novel emphasis on activity recognition to improve predictive accuracy. The method utilizes a multi-camera system to extract 2D joint locations, which are then fused using a particle filter technique for 3D pose generation. Using 3D data, deep learning models are developed to first recognize the activity class, and then take it as auxiliary information for predicting the motion trajectory. Through a comprehensive experiment, we evaluated the proposed methodology. While the main innovation of the proposed approach lies in the incorporation of deep learning-based activity recognition into the trajectory prediction system, the experiment results revealed the activity-aware system's capability to enhance prediction performance by a minimum of 6.4% and up to 16.6% in short-term forecasts, in compare with a conventional approach. Additionally, we analyzed the effects of varying time windows and joint selections on predictive outcomes across diverse scenarios and discussed the implications of these findings. By enhancing the prediction of human motion, this approach holds promise in improving workspace safety while encouraging effective interactions within complex environments. [ABSTRACT FROM AUTHOR]