Search

Heart sound auscultation has been applied in clinical usage for early screening of cardiovascular diseases. Due to the high demand for auscultation expertise, automatic auscultation can help with auxiliary diagnosis and reduce the burden of training professional clinicians. Nevertheless, there is a limit to classic machine learning's performance improvement in the era of Big Data. Deep learning has outperformed classic machine learning in many research fields, as it employs more complex model architectures with a stronger capability of extracting effective representations. Moreover, it has been successfully applied to heart sound analysis in the past years. As most review works about heart sound analysis were carried out before 2017, the present survey is the first to work on a comprehensive overview to summarise papers on heart sound analysis with deep learning published in 2017–2022. This work introduces both classic machine learning and deep learning for comparison, and further offer insights about the advances and future research directions in deep learning for heart sound analysis. [ABSTRACT FROM AUTHOR]

Deep learning methods have advanced quickly in brain imaging analysis over the past few years, but they are usually restricted by the limited labeled data. Pre-trained model on unlabeled data has presented promising improvement in feature learning in many domains, such as natural language processing. However, this technique is under-explored in brain network analysis. In this paper, we focused on pre-training methods with Transformer networks to leverage existing unlabeled data for brain functional network classification. First, we proposed a Transformer-based neural network, named as BrainNPT, for brain functional network classification. The proposed method leveraged token as a classification embedding vector for the Transformer model to effectively capture the representation of brain networks. Second, we proposed a pre-training framework for BrainNPT model to leverage unlabeled brain network data to learn the structure information of brain functional networks. The results of classification experiments demonstrated the BrainNPT model without pre-training achieved the best performance with the state-of-the-art models, and the BrainNPT model with pre-training strongly outperformed the state-of-the-art models. The pre-training BrainNPT model improved 8.75% of accuracy compared with the model without pre-training. We further compared the pre-training strategies and the data augmentation methods, analyzed the influence of the parameters of the model, and explained the trained model.

Early diagnosis and intervention of depression promote complete recovery, with its traditional clinical assessments depending on the diagnostic scales, clinical experience of doctors and patient cooperation. Recent researches indicate that functional near-infrared spectroscopy (fNIRS) based on deep learning provides a promising approach to depression diagnosis. However, collecting large fNIRS datasets within a standard experimental paradigm remains challenging, limiting the applications of deep networks that require more data. To address these challenges, in this paper, we propose an fNIRS-driven depression recognition architecture based on cross-modal data augmentation (fCMDA), which converts fNIRS data into pseudo-sequence activation images. The approach incorporates a time-domain augmentation mechanism, including time warping and time masking, to generate diverse data. Additionally, we design a stimulation task-driven data pseudo-sequence method to map fNIRS data into pseudo-sequence activation images, facilitating the extraction of spatial-temporal, contextual and dynamic characteristics. Ultimately, we construct a depression recognition model based on deep classification networks using the imbalance loss function. Extensive experiments are performed on the two-class depression diagnosis and five-class depression severity recognition, which reveal impressive results with accuracy of 0.905 and 0.889, respectively. The fCMDA architecture provides a novel solution for effective depression recognition with limited data.

Brain-computer interfaces (BCIs) can enable direct communication with assistive devices by recording and decoding signals from the brain. To achieve high performance, many electrodes will be used, such as the recently developed invasive BCIs with channel numbers up to hundreds or even thousands. For those high-throughput BCIs, channel selection is important to reduce signal redundancy and invasiveness while maintaining decoding performance. However, such endeavour is rarely reported for invasive BCIs, especially those using deep learning methods. Two deep learning-based methods, referred to as Gumbel and STG, were proposed in this paper. They were evaluated using the Stereo-electroencephalography (SEEG) signals, and compared with three other methods, including manual selection, mutual information-based method (MI), and all channels (all channels without selection). The task is to classify the SEEG signals into five movements using channels selected by each method. When 10 channels were selected, the mean classification accuracies using Gumbel, STG (referred to as STG-10), manual selection, and MI selection were 65%, 60%, 60%, and 47%, respectively, whilst the accuracy was 59% using all channels (no selection). In addition, an investigation of the selected channels showed that Gumbel and STG have successfully identified the pre-central and post-central areas, which are closely related to motor control. Both Gumbel and STG successfully selected the informative channels in SEEG recordings while maintaining decoding accuracy. This study enables future high-throughput BCIs using deep learning methods, to identify useful channels and reduce computing and wireless transmission pressure.

The olfactory system enables humans to smell different odors, which are closely related to emotions. The high temporal resolution and non-invasiveness of Electroencephalogram (EEG) make it suitable to objectively study human preferences for odors. Effectively learning the temporal dynamics and spatial information from EEG is crucial for detecting odor-induced emotional valence. In this paper, we propose a deep learning architecture called Temporal Attention with Spatial Autoencoder Network (TASA) for predicting odor-induced emotions using EEG. TASA consists of a filter-bank layer, a spatial encoder, a time segmentation layer, a Long Short-Term Memory (LSTM) module, a multi-head self-attention (MSA) layer, and a fully connected layer. We improve upon the previous work by utilizing a two-phase learning framework, using the autoencoder module to learn the spatial information among electrodes by reconstructing the given input with a latent representation in the spatial dimension, which aims to minimize information loss compared to spatial filtering with CNN. The second improvement is inspired by the continuous nature of the olfactory process; we propose to use LSTM-MSA in TASA to capture its temporal dynamics by learning the intercorrelation among the time segments of the EEG. TASA is evaluated on an existing olfactory EEG dataset and compared with several existing deep learning architectures to demonstrate its effectiveness in predicting olfactory-triggered emotional responses. Interpretability analyses with DeepLIFT also suggest that TASA learns spatial-spectral features that are relevant to olfactory-induced emotion recognition.

Automatic identification of visual learning style in real time using raw electroencephalogram (EEG) is challenging. In this work, inspired by the powerful abilities of deep learning techniques, deep learning-based models are proposed to learn high-level feature representation for EEG visual learning identification. Existing computer-aided systems that use electroencephalograms and machine learning can reasonably assess learning styles. Despite their potential, offline processing is often necessary to eliminate artifacts and extract features, making these methods unsuitable for real-time applications. The dataset was chosen with 34 healthy subjects to measure their EEG signals during resting states (eyes open and eyes closed) and while performing learning tasks. The subjects displayed no prior knowledge of the animated educational content presented in video format. The paper presents an analysis of EEG signals measured during a resting state with closed eyes using three deep learning techniques: Long-term, short-term memory (LSTM), Long-term, short-term memory-convolutional neural network (LSTM-CNN), and Long-term, short-term memory-Fully convolutional neural network (LSTM-FCNN). The chosen techniques were based on their suitability for real-time applications with varying data lengths and the need for less computational time. The optimization of hypertuning parameters has enabled the identification of visual learners through the implementation of three techniques. LSTM-CNN technique has the highest average accuracy of 94%, a sensitivity of 80%, a specificity of 92%, and an F1 score of 94% when identifying the visual learning style of the student out of all three techniques. This research has shown that the most effective method is the deep learning-based LSTM-CNN technique, which accurately identifies a student's visual learning style.

Sleep staging serves as a fundamental assessment for sleep quality measurement and sleep disorder diagnosis. Although current deep learning approaches have successfully integrated multimodal sleep signals, enhancing the accuracy of automatic sleep staging, certain challenges remain, as follows: 1) optimizing the utilization of multi-modal information complementarity, 2) effectively extracting both long- and short-range temporal features of sleep information, and 3) addressing the class imbalance problem in sleep data. To address these challenges, this paper proposes a two-stream encode-decoder network, named TSEDSleepNet, which is inspired by the depth sensitive attention and automatic multi-modal fusion (DSA2F) framework. In TSEDSleepNet, a two-stream encoder is used to extract the multiscale features of electrooculogram (EOG) and electroencephalogram (EEG) signals. And a self-attention mechanism is utilized to fuse the multiscale features, generating multi-modal saliency features. Subsequently, the coarser-scale construction module (CSCM) is adopted to extract and construct multi-resolution features from the multiscale features and the salient features. Thereafter, a Transformer module is applied to capture both long- and short-range temporal features from the multi-resolution features. Finally, the long- and short-range temporal features are restored with low-layer details and mapped to the predicted classification results. Additionally, the Lovász loss function is applied to alleviate the class imbalance problem in sleep datasets. Our proposed method was tested on the Sleep-EDF-39 and Sleep-EDF-153 datasets, and it achieved classification accuracies of 88.9% and 85.2% and Macro-F1 scores of 84.8% and 79.7%, respectively, thus outperforming conventional traditional baseline models. These results highlight the efficacy of the proposed method in fusing multi-modal information. This method has potential for application as an adjunct tool for diagnosing sleep disorders.

Computational biomechanical analysis plays a pivotal role in understanding and improving human movements and physical functions. Although physics-based modeling methods can interpret the dynamic interaction between the neural drive to muscle dynamics and joint kinematics, they suffer from high computational latency. In recent years, data-driven methods have emerged as a promising alternative due to their fast execution speed, but label information is still required during training, which is not easy to acquire in practice. To tackle these issues, this paper presents a novel physics-informed deep learning method to predict muscle forces without any label information during model training. In addition, the proposed method could also identify personalized muscle-tendon parameters. To achieve this, the Hill muscle model-based forward dynamics is embedded into the deep neural network as the additional loss to further regulate the behavior of the deep neural network. Experimental validations on the wrist joint from six healthy subjects are performed, and a fully connected neural network (FNN) is selected to implement the proposed method. The predicted results of muscle forces show comparable or even lower root mean square error (RMSE) and higher coefficient of determination compared with baseline methods, which have to use the labeled surface electromyography (sEMG) signals, and it can also identify muscle-tendon parameters accurately, demonstrating the effectiveness of the proposed physics-informed deep learning method.

Major Depression Disorder (MDD) is a common yet destructive mental disorder that affects millions of people worldwide. Making early and accurate diagnosis of it is very meaningful. Recently, EEG, a non-invasive technique of recording spontaneous electrical activity of brains, has been widely used for MDD diagnosis. However, there are still some challenges in data quality and data size of EEG: (1) A large amount of noise is inevitable during EEG collection, making it difficult to extract discriminative features from raw EEG; (2) It is difficult to recruit a large number of subjects to collect sufficient and diverse data for model training. Both of the challenges cause the overfitting problem, especially for deep learning methods. In this paper, we propose DiffMDD, a diffusion-based deep learning framework for MDD diagnosis using EEG. Specifically, we extract more noise-irrelevant features to improve the model's robustness by designing the Forward Diffusion Noisy Training Module. Then we increase the size and diversity of data to help the model learn more generalized features by designing the Reverse Diffusion Data Augmentation Module. Finally, we re-train the classifier on the augmented dataset for MDD diagnosis. We conducted comprehensive experiments to test the overall performance and each module's effectiveness. The framework was validated on two public MDD diagnosis datasets, achieving the state-of-the-art performance.

The document is the table of contents for the May 2024 issue of the IEEE Transactions on Semiconductor Manufacturing. It includes papers on various topics such as yield modeling, analysis, and enhancement, advanced processing, advanced process control, equipment and automation technology, non-silicon materials, emerging areas, and environment, safety, and health. The articles cover subjects such as chip-scale chemical mechanical polishing, threshold voltage deviation monitoring, curvilinear standard cell design, fabrication of silicon nanocone arrays, improving the reliability of through silicon vias, energy consumption and carbon emission reduction in HVAC systems, low-k silicon dioxide synthesis, chemical mechanical polishing of diamond epitaxial layers, conditional variable selection based on deep learning, and eco-friendly dry-cleaning and diagnostics of silicon dioxide deposition chambers. The document also includes an announcement for the call for nominations for the 2024 IEEE EDS Early Career Award. [Extracted from the article]

Recent developments in the field of non-local attention (NLA) have led to a renewed interest in self-similarity-based single image super-resolution (SISR). Researchers usually use the NLA to explore non-local self-similarity (NSS) in SISR and achieve satisfactory reconstruction results. However, a surprising phenomenon that the reconstruction performance of the standard NLA is similar to that of the NLA with randomly selected regions prompted us to revisit NLA. In this paper, we first analyzed the attention map of the standard NLA from different perspectives and discovered that the resulting probability distribution always has full support for every local feature, which implies a statistical waste of assigning values to irrelevant non-local features, especially for SISR which needs to model long-range dependence with a large number of redundant non-local features. Based on these findings, we introduced a concise yet effective soft thresholding operation to obtain high-similarity-pass attention (HSPA), which is beneficial for generating a more compact and interpretable distribution. Furthermore, we derived some key properties of the soft thresholding operation that enable training our HSPA in an end-to-end manner. The HSPA can be integrated into existing deep SISR models as an efficient general building block. In addition, to demonstrate the effectiveness of the HSPA, we constructed a deep high-similarity-pass attention network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive experimental results demonstrate that HSPAN outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code and a pre-trained model were uploaded to GitHub (https://github.com/laoyangui/HSPAN) for validation. [ABSTRACT FROM AUTHOR]

Efficient point cloud compression is essential for applications like virtual and mixed reality, autonomous driving, and cultural heritage. This paper proposes a deep learning-based inter-frame encoding scheme for dynamic point cloud geometry compression. We propose a lossy geometry compression scheme that predicts the latent representation of the current frame using the previous frame by employing a novel feature space inter-prediction network. The proposed network utilizes sparse convolutions with hierarchical multiscale 3D feature learning to encode the current frame using the previous frame. The proposed method introduces a novel predictor network for motion compensation in the feature domain to map the latent representation of the previous frame to the coordinates of the current frame to predict the current frame’s feature embedding. The framework transmits the residual of the predicted features and the actual features by compressing them using a learned probabilistic factorized entropy model. At the receiver, the decoder hierarchically reconstructs the current frame by progressively rescaling the feature embedding. The proposed framework is compared to the state-of-the-art Video-based Point Cloud Compression (V-PCC) and Geometry-based Point Cloud Compression (G-PCC) schemes standardized by the Moving Picture Experts Group (MPEG). The proposed method achieves more than 88% BD-Rate (Bjøntegaard Delta Rate) reduction against G-PCCv20 Octree, more than 56% BD-Rate savings against G-PCCv20 Trisoup, more than 62% BD-Rate reduction against V-PCC intra-frame encoding mode, and more than 52% BD-Rate savings against V-PCC P-frame-based inter-frame encoding mode using HEVC. These significant performance gains are cross-checked and verified in the MPEG working group. [ABSTRACT FROM AUTHOR]

Thanks to the many submissions we have received, we can present this second part of our special issue on "AI-eXplained." In here, we continue our mission to demystify the intricate world of artificial intelligence and make it accessible to a broader audience. As AI continues to evolve and integrate into various aspects of our lives, it becomes increasingly important to bridge the gap between experts and those eager to understand the inner workings of AI systems. Our goal remains unchanged—to present AI concepts in a comprehensible, engaging, and interactive manner, empowering our readers to explore and grasp the enchanting world of AI. [ABSTRACT FROM AUTHOR]

Soaring energy demand in power sector demands an intelligent electricity grid which is able to meet the energy requirements of consumers with higher efficiency and reliability. This paper discusses about how Smart Grid (SG) is the solution for such requirements in the energy sector. SG constitutes of three main components: Information Technology (IT), Operational Technology (OT), and Advanced Metering Infrastructure (AMI). It is important to synchronize these three components in order to seamlessly deliver electricity to the consumers connected to the electric network. Cyberattack is increasing sharply in the SG infrastructure and triggers disrupting or malicious manipulation of entire electricity network. It is crucial to maintain the data traffic, power transmission and distribution with high reliability. This paper aims at arriving at the taxonomy of attacks that are subjected to IT, OT and AMI components of SG. This exhaustive taxonomy further opens avenues to evolve mitigation strategies for these attacks which will eventually aid in SG protection. Elaborate discussion about various mitigation strategies for various types of attacks is performed. The systematic approach of methods and techniques along with the type of solution it provides to SG applications is discussed. It is evident from literature that Machine Learning (ML), Deep Learning (DL) and signal processing techniques are very efficient and robust in attack detection and retaliating in SG. ML model pro. Hence the importance of framework and tools in SG environment is analyzed. A case study is also presented that emphasizes on SG Attacks on the specific system and its mitigation strategies.

Early fault warning for large-scale high-speed rotating machinery can effectively reduce unplanned downtime and avoid major safety accidents. Aiming at the problems of difficult screening of multi-source common sensitive features, the challenging training of neural networks with a small number of sensitive features, and the difficulty of directly using generative adversarial networks for early fault warning, this paper constructs an early fault warning model based on multi-source common sensitive features and an improved Wasserstein generative adversarial network, proposing an early fault warning method for rotating machinery. The model was verified by using the open XJTU-SY bearing laboratory data, the P3409A centrifugal pump bearing fault engineering case data of a petrochemical company and the rotor system engineering case data of a circulating hydrogen centrifugal compressor of a petrochemical company. The early fault warning method of rotating machinery proposed in this paper warns the bearing fault of centrifugal pump 160 hours in advance and the rotor system fault of centrifugal compressor 1330 minutes in advance. Compared with the two published methods, the proposed method has better early fault warning effect, better normal and abnormal health index discrimination and less false warning.

With the widespread availability and utilization of 3D meshes across various applications, the need for accurately assessing their visual quality has become increasingly important. Despite the significance of this task, the literature offers few No-Reference (NR) approaches for evaluating the visual quality of 3D meshes. In response to this gap, this paper proposes a novel NR approach tailored specifically to score the quality of 3D meshes. After rendering a 3D mesh into 2D views and patches, a pre-trained convolutional neural network automatically extracts deep features from. These features are then employed in a Multi-Layer Perceptron regressor to predict the quality score of the rendered images. The obtained scores are combined with their corresponding BRISQUE scores, and an additional MLP regressor is used to predict the final score. We present experimental results demonstrating the effectiveness and robustness of our approach across a diverse range of 3D mesh datasets. Comparative analyses with existing NR methods underscore the superior performance and versatility of the proposed approach. Overall, this paper contributes to the advancement of NR techniques for assessing 3D mesh quality, offering a valuable tool for researchers, practitioners, and developers working with 3D models across various domains.

Recently, the combination of convolutional neural network (CNN) and long short-term memory (LSTM) exhibits better performance than single network architecture. Most of these studies connect LSTM networks behind CNNs. When operating on hardware, the current design of CNN-LSTM is similar to a pipeline architecture. However, the classic structure lead to a feature loss when data is sent to LSTM since CNN is not good at extracting temporal features. At the same time, as the depth and scale increases, it will bring a huge amount of computation, which makes hardware implementation difficult. Based on that, a parallel CNN-LSTM architecture is proposed, in which two networks extract features from the input data synchronously, being proven to be more effective than classical CNN-LSTM. This paper designs a parallel CNN-LSTM computing device based on FPGA. The device is divided into control unit and operation unit. Control stream and data stream transport between the two units, ensuring the proper running of the device. A highly parallel multi-channel convolution layer and pooling layer are designed to improve the calculation efficiency. A 4-stage pipeline structure is adopted to implement the LSTM part. This paper makes full use of on-chip BRAM to design a look-up table for activation function approximation, reducing the resource consumption by 95% compared with the traditional polynomial approximation. Finally, we verify our device under cooperative spectrum sensing (CSS) and handwritten classification scenarios. Proposed device reaches higher accuracy in two scenarios compared with classic CNN-LSTM structure as well as faster calculating speed, and the overall project power is limited below 2W. The scalability and limitation of this computing device are also discussed.

Reinforcement learning (RL) has seen an uptick in research interest in recent years, with many papers published in a plethora of different fields, topics and applications. A lot of that can be attributed to the recent advancements in machine learning (ML) and deep learning (DL) as a whole, the power of deep neural networks and the incorporation of them into reinforcement learning algorithms and techniques. This marriage of deep neural networks and traditional reinforcement learning methods has enabled reinforcement learning algorithms to achieve incredible results in use-cases that were previously deemed to be too complex. One of these highly complex scenarios is the realm of flight. In the last few years the number of papers published in this field has increased dramatically, but there is a fairly big divide between two major categories (quadcopters vs. fixed-wing aircraft). There is published work focusing on reinforcement learning using quadcopters, also known as drones, and there is published work focusing on fixed-wing aircraft, the more traditional aircraft, commonly referred to as airplanes. This paper focuses on the latter and will give an overview over the current state of reinforcement learning in the field of fixed-wing aviation. It will look into available toolkits usable for research as well as give an overview of different fields and tasks within reinforcement learning for fixed-wing aircraft control tasks. It contributes by providing detailed information about the methodologies used in each paper, while pointing out each paper’s contributions as well. A total of 48 papers on fixed-wing RL and airplane control are discussed. With the main focus lying on papers published post 2015, after the publication of Mnih et al.’s impactful deep RL paper.

The convergence of Information Technology (IT), Operational Technology (OT), and Educational Technology (ET) has led to the emergence of the fourth industrial revolution. As a result, a new concept has emerged known as Digital Twins (DT), which is defined as “a virtual representation of various objects or systems that receive data from physical objects/systems to make changes and corrections”. In the aviation industry, numerous attempts have been made to utilize DT in the design, manufacturing, and condition monitoring of aircraft fleets. Among these research efforts, real-time, accurate, fast, and predictive condition monitoring methods play a crucial role in ensuring the safe and efficient performance of aircraft. Using DT for condition and fleet monitoring not only enhances the reliability and safety of aircraft but also reduces operational and maintenance costs. In this paper, the conducted studies on the applications of DT systems for condition monitoring of aircraft units and the aerospace sector are discussed and reviewed. The aim of this review paper is to analyse the current developments of DT systems in the aviation industry as well as explain the remaining challenges of DT systems. Then Finally, future trends of DT systems along with aircraft are presented. Among reviewed papers, most of them have used computational fluid dynamics, finite element methods, and artificial intelligence techniques for developing DT models for aircraft. At the same time, most of these analyses are dedicated to the failure and crack detection body of aircraft as well as engine fault detection. Life prediction is another popular application for using DT in aircraft units that could help the engineers predict the maintenance required for different parts of the aircraft. Finally, the application of DT in marine, power systems, and space programs has been also reviewed and the lessons learned from them have been translated to the aviation sector.

Sentiment analysis extracts valuable insights from textual sources using computation, textual or systematic analysis, and natural language processing. It identifies and measures the attitudes, beliefs, and emotional states individuals express through text data. Recent research on sentiment analysis has largely focused on the English language; therefore, low-resource languages are getting much less attention. Conducting sentiment analysis of low-resource languages is difficult because large datasets and related repositories are unavailable. This paper creates a new dataset for low-resource language (Urdu) to address this issue. The dataset, namely LUCSA-23, consists of more than 65,000 user reviews from various genres, including food, sports, showbiz, apps, and political reviews from developing countries, i.e., Pakistan. Urdu domain experts further annotate the created dataset. This paper proposes an Urdu sentiment analysis approach leveraging the transformer model, i.e., XLM-R and GPT-2. It preprocesses the Urdu text input, generates BERT embeddings, and passes them to the proposed classifier as input for sentiment classification. The proposed classifier is compared with machine/deep/embedded classifiers to evaluate its performance. The findings show that the proposed classifiers outperform existing state-of-the-art approaches with an accuracy of 95%.

This paper introduces a novel method for accurately estimating the 2D coordinates of hand keypoints from single static images, utilizing a sequential convolutional neural network optimized with Maximum Likelihood Estimation Loss. Unlike traditional heatmap-based techniques, our approach eliminates the need to generate label heatmaps and sidesteps the direct optimization of model parameters based on noisy labels. Instead, it concentrates on modeling the distribution of the discrepancies between predicted results and ground truth, rather than the potential presence of noisy labels, thus enabling the direct prediction of hand keypoint coordinates. Furthermore, we propose a sequential training and inference framework that consists of a deep convolutional backbone network and a multi-stage sequential network. Each stage of this network features similar structures, facilitating the progressive and precise prediction of hand keypoint coordinates. Our extensive experimental results demonstrate that our approach is both highly accurate and robust, outperforming mainstream methods under the experimental conditions detailed in this paper.

Automated early detection and classification of paddy diseases help in applying treatment efficiently according to the detected diseases. Early detection also minimises the usage of chemical substances and pesticides and hinders the spread of the disease to healthy crops. On a broader scale, it aids in halting the global spread of diseases. Thus, it ultimately promotes healthier rice crops and increased yield. In this survey paper, we present a thorough exploration of deep learning (DL) models for the classification of paddy diseases. Our paper delves into the motivation behind this research study, reveals different paddy diseases and their associated symptoms, and unravels various deep-learning models employed for disease detection. We have also discussed strategies used by researchers for improving the performance of DL models, along with adaptations tailored for application-specific contexts. Additionally, we illustrate relevant research findings, explore datasets utilised in this domain, and analyse approaches for data augmentation. Through an exhaustive investigation, we emphasise existing research gaps, challenges, and open issues, concluding in a discussion on avenues for future exploration.

The transformer architecture has been focused on many tasks like natural language processes, vision tasks and etc. The most important and general requirement of using the transformer-based architecture is that the model must be trained on a large-scale dataset before it can be fine-tuned for a specific task like classification, object detection and etc. However, in this paper, we find that the transformer architecture has better generalization capability to capture the features from data samples for sleep stage classification than CNN-based architectures, despite using a small-scale dataset without pretraining on large-scale dataset. This outcome contradicts the widely-held belief that a transformer architecture is more effective when trained on large datasets. In this paper, we investigate the attention behavior of a transformer model and demonstrate how global and local attentions influence an attention map in a transformer architecture. Finally, through experiments, we show that restricting global attention using ‘Masked Multi-Head Self-Attention (M-MHSA)’ can lead to improved model generalization in sleep stage classification compared with the previous methodologies and original transformer-based architecture on three different datasets.

Time series forecasting involves justifying assertions scientifically regarding potential states or predicting future trends of an event based on historical data recorded at various time intervals. The field of time series forecasting, supported by diverse deep learning models, has made significant advancements, rendering it a prominent research area. The broad spectra of available time series datasets serve as valuable resources for conducting extensive studies in time series analysis with varied objectives. However, the complexity and scale of time series data present challenges in constructing reliable prediction models. In this paper, our objectives are to introduce and review methodologies for modeling time series data, outline the commonly used time series forecasting datasets and different evaluation metrics. We delve into the essential architectures for trending an input dataset and offer a comprehensive assessment of the recently developed deep learning prediction models. In general, different models likely serve different design goals. We boldly examine the performance of these models under the same time series input dataset with an identical hardware computing system. The measured performance may reflect the design flexibility among all the ranked models. And through our experiments, the SCINet model performs the best in accuracy with the ETT energy input dataset. The results we obtain could give a glimpse in understanding the model design and performance relationship. Upon concluding the paper, we shall provide further discussion on future deep learning research directions in the realm of time series forecasting.

This paper proposes to explore the Radio Frequency Fingerprint (RFF) identification with a virtual database generator. RFF is a unique signature created in the emitter transmission chain by the hardware impairments. These impairments may be used as a secure identifier as they cannot be easily replicated for spoofing purposes. In recent years, the RFF identification relies mainly on Deep Learning (DL), and large databases are consequently needed to improve identification in different environmental conditions. In this paper, we introduce the so-called RiFyFi_VDG, referring to Radio Frequency Fingerprint Virtual Database Generator, and explore individually the impairment impact on the classification accuracy to highlight the most relevant impairment. Different transmission scenarios are then explored, such as the impact of the data type (being a preamble or a payload) and the data size. Design rules of real databases are finally drawn for the different scenarios. We found out that the power amplifier imperfections play the biggest role in RFF accuracy and that average accuracy levels of 94% can be reached when combining the various hardware impairments at the transmitter.

With the rapid development of the Internet, people independently write comments with emotional characteristics on e-commerce platforms, which express consumers’ emotional tendencies towards products or services from multiple perspectives. The sentiment analysis technology of product reviews has attracted more and more attention. In recent years, the Transformer model has performed well in the field of text sentiment analysis. However, in the process of text emotion classification, the Transformer basic model cannot be well obtained when the distance between words is relatively long, and there are problems such as a low accuracy rate and recall rate and too much time spent on the overall training of model construction. Using conditional random field (CRF) as a classifier can effectively solve the problem of too long text segmentation distance, reduce model training time, and increase the accuracy of text sentiment analysis. Therefore, this paper proposes a new method for text sentiment analysis combining the improved Transformer model and a conditional random field. First of all, this paper enhances the Transformer model by adapting the decoder to better suit the task of sentiment classification, and introduces an enhanced version of the Transformer model. It is then classified by combining long-short-term memory (LSTM) and CRF. The experimental results show that in the IMDB data set, the Transformer CRF model has higher accuracy, recall rate, and F1 value, reaching 85.51%, 83.77%, and 85.06%, respectively. Compared with other methods, the results of evaluation indicators further verify that the text method has better recognition performance and generalization ability, and at the same time, it has a better understanding of customers’ emotions towards a specific aspect, provides users with accurate services, effectively improves user satisfaction, and has a high use value for enterprises’ business decisions.

This paper introduces an innovative methodology to disrupt deep-learning (DL) surveillance systems by implementing an adversarial framework strategy, inducing misclassification in live video objects and extending attacks to real-time models. Focusing on the vulnerability of image-categorization models, the study evaluates the effectiveness of face mask surveillance against adversarial threats. A real-time system, employing the ShuffleNet V1 transfer-learning algorithm, was trained on a Kaggle dataset for face mask detection accuracy. Using a white-box Fast Gradient Sign Method (FGSM) attack with epsilon at 0.13, the study successfully generated adversarial frames, deceiving the face mask detection system and prompting unintended video predictions. The findings highlight the risks posed by adversarial attacks on critical video surveillance systems, specifically those designed for face mask detection. The paper emphasizes the need for proactive measures to safeguard these systems before real-world deployment, crucial for ensuring their robustness and reliability in the face of potential adversarial threats.

Situation Awareness (SA) is a process of sensing, understanding and predicting the environment and is an important component in complex systems. The reception of information from the environment tends to be continuous and of a multimodal nature. AI technologies provide a more efficient and robust support by subdividing the different stages of SA objectives into tasks such as data fusion, representation, classification, and prediction. This paper provides an overview of AI and multimodal methods used to build, enhance and evaluate SA in a variety of environments and applications. Emphasis is placed on enhancing perceptual integrity and persistence. Research indicates that the integration of artificial intelligence and multimodal approaches has significantly enhanced perception and comprehension in complex systems. However, there remains a research gap in projecting future situations and effectively fusing multimodal information. This paper summarizes some of the use cases and lessons learned where AI and multimodal techniques have been used to deliver SA. Future perspectives and challenges are proposed, including more comprehensive predictions, greater interpretability, and more advanced visual information.

This paper presents the 6th edition of the “Drone-vs-Bird” detection challenge, jointly organized with the WOSDETC workshop within the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2023. The main objective of the challenge is to advance the current state-of-the-art in detecting the presence of one or more Unmanned Aerial Vehicles (UAVs) in real video scenes, while facing challenging conditions such as moving cameras, disturbing environmental factors, and the presence of birds flying in the foreground. For this purpose, a video dataset was provided for training the proposed solutions, and a separate test dataset was released a few days before the challenge deadline to assess their performance. The dataset has continually expanded over consecutive installments of the Drone-vs-Bird challenge and remains openly available to the research community, for non-commercial purposes. The challenge attracted novel signal processing solutions, mainly based on deep learning algorithms. The paper illustrates the results achieved by the teams that successfully participated in the 2023 challenge, offering a concise overview of the state-of-the-art in the field of drone detection using video signal processing. Additionally, the paper provides valuable insights into potential directions for future research, building upon the main pros and limitations of the solutions presented by the participating teams.

Diabetic Retinopathy is one of the leading global causes of vision impairment and blindness in humans. It has seen a rise in prevalence, necessitating the development of advanced automatic detection methods. This paper presents a survey of the evolution in deep learning techniques for diabetic retinopathy detection, emphasizing the transition from traditional machine learning to sophisticated deep learning architectures such as convolutional neural networks. It discusses the role of transfer learning, end-to-end learning, and hybrid models in overcoming medical detection challenges while highlighting the need for artificial intelligence interpretability and real-time screening integration in clinical workflows. Building on this survey, the paper introduces a focused study on cross-dataset deployment of transfer learning for diabetic retinopathy detection and grading. Consequently, this paper evaluates 26 pre-trained models from various convolutional neural network families to provide a comprehensive comparison between the state-of-the-art CNN architectures in the field. Additionally, this study also employs Grad-CAM visualization to interpret the model’s decision-making, bridging advanced artificial intelligence techniques with practical healthcare applications for diabetic retinopathy.

Researchers are working towards automation of ontology construction to manage the ever-growing data on the web. Currently, there is a shift from the use of machine learning techniques towards exploration of deep learning models for ontology construction. Deep learning model are capable of extracting terms, entities, relations, and classifications, and perform axiom learning from the underutilized richness of web-based knowledge. There has been remarkable progress in automatic ontology creation using deep learning models since they can perform word embedding, long-term dependency acquisition, concept extraction from large corpora, and inference of abstracted relationships based on broad corpora. Despite their emerging importance, deep learning models remain underutilized in ontology construction, and there is no comprehensive review of their application in ontology learning. This paper presents a comprehensive review of existing deep-learning models for the construction of ontologies, the strength and the weaknesses presented by the deep learning models for ontology learning as well as promising directions to achieve a more robust deep learning models. The Deep Learning models reviewed include Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Long-Short Term Memory (LSTMs), and Gated Recurrent Unit (GRU) as well as their ensembles. While these traditional deep learning models have achieved great success, one of their limitations is that they struggle to understand the meaning and order of data in sequences. CNNs and RNN-based models such as LSTMs and GRUs can be computationally expensive due to their large number of parameters or complex gating mechanisms. Furthermore, RNN models suffer from vanishing gradients, making it difficult to learn long-term relationships in sequences. Additionally, RNN-based models process information sequentially, limiting their ability to take advantage of powerful parallel computing hardware, slowing down training and inference, especially for long sequences. Consequently, there has been a shift towards Generative Pre-Trained (GPT) models and Bidirectional Encoder Representations from Transformers (BERT) models. This paper also reviewed the GPT-3, GPT-4, and the BERT models for extracting terms, entities, relations, and classifications. While GPT models excel in contextual understanding and flexibility, they fall short when handling domain-specific terminology and disambiguating complex relationships. Fine-tuning and domain-specific training data could minimize these shortcomings, and further enhance the performance of GPT in term and relation extraction tasks. On the other hand, the BERT models excel in comprehending context-heavy texts, but struggles with higher-level abstraction and inference tasks due to a lack of explicit semantic knowledge, thus necessitating inference for unspecified relationships. The paper recommends further research on deep learning models for ontology alignment and merging. Also, the ensembling of deep learning models and the use of domain-specific knowledge for ontology learning require further research for ontology construction.

Nowadays, the advancements of wearable consumer devices have become a predominant role in healthcare gadgets. There is always a demand to obtain robust recognition of heterogeneous human activities in complicated IoT environments. The knowledge attained using these recognition models will be then combined with healthcare applications. In this way, the paper proposed a novel deep learning framework to recognize heterogeneous human activities using multimodal sensor data. The proposed framework is composed of four phases: employing dataset and processing, implementation of deep learning model, performance analysis, and application development. The paper utilized the recent KU-HAR database with eighteen different activities of 90 individuals. After preprocessing, the hybrid model integrating Extreme Learning Machine (ELM) and Gated Recurrent Unit (GRU) architecture is used. An attention mechanism is then included for further enhancing the robustness of human activity recognition in the IoT environment. Finally, the performance of the proposed model is evaluated and comparatively analyzed with conventional CNN, LSTM, GRU, ELM, Transformer and Ensemble algorithms. To the end, an application is developed using the Qt framework which can be deployed on any consumer device. In this way, the research sheds light on monitoring the activities of critical patients by healthcare professionals remotely. The proposed ELM-GRUaM model achieved supreme performance in recognizing multimodal human activities with an overall accuracy of 96.71% as compared with existing models.

Several recent studies in dentistry and maxillofacial imaging have concentrated on the Mandibular Canal (MC) segmentation in digital dental radiographs. In this research domain, deep learning approaches have demonstrated promising outcomes. This systematic literature review (SLR) aims to comprehensively analyze and synthesize the recent advancements in applying deep learning techniques for MC segmentation in digital dental radiographs. This review encompasses studies published between 2018 and 2023, sourced from reputable databases, including PubMed, ScienceDirect, IEEE Xplore, Google Scholar, and SpringerLink. This study identified 30 primary research papers focusing on MC segmentation in digital dental radiographs. The review categorizes papers into two groups based on the digital dental radiograph type. The first group uses a 2D digital dental radiograph from a panoramic radiograph and 2D Cone Beam Computed Tomography (CBCT) scans. The second group uses 3D datasets from volumetric data from CBCT scans. The synthesized knowledge from this review is intended to guide researchers, dentists, and oral surgeons in leveraging deep learning advancements in MC segmentation for oral and maxillofacial surgery. Prior studies have faced challenges, including limited public datasets, variations in MC anatomy, time consumption and observer variability in MC annotation, complexities in deep learning models, and lack of practical implementation. To overcome these challenges, it is suggested that more public datasets be collaboratively created and shared within the research community, focusing on handling anatomy variability, improving digital dental radiograph quality, streamlining annotation processes through automated tools, and simplifying deep learning models for practical implementation.

The relentless global population growth and the ever-increasing food demand pose formidable challenges to the agricultural sector. Farmers grapple with the ongoing challenge of wildlife-induced crop damage and human loss, which not only impedes food production but also exacerbates supply and demand imbalances. However, the rise of TinyML enables Edge AI as a promising avenue for implementing resource-efficient deep learning techniques on low-end edge devices. In this paper, we introduce an innovative solution that harnesses the power of Edge AI using tinyML-based deep learning algorithms in conjunction with the Internet of Things (IoT) for animal intrusion detection and deterrence system. The proposed system is developed to create remotely managed defense system tailored to safeguard vast agricultural expanses. It integrates a laser detection system and an AI-CAM with light weight deep learning algorithms for animal intrusion detection and classification. This system also ensures efficient animal deterrence and real-time monitoring for farmers, enabling them to assess the situation with the assistance of an intelligent rover build using IoT. This work emphasizes on proposing a light-weight deep learning model named EvoNet for animal classification. Results reveal that the proposed model achieves the highest accuracy at 96.7%, surpassing other models presented in this paper. However, for edge devices where compact file sizes are crucial, the model also offers comparable accuracy with file sizes as low as 1.63MB with the help of pruning and quantization techniques. This conceptualized solution has the potential to revolutionize agricultural wildlife management, ushering in a new era of crop protection and economic resilience.

Billions of people globally use Android devices (https://backlinko.com/iphone-vs-android-statistics). As such, these devices are highly targeted by security attackers. One of the most threatening attacks is to infect devices with malicious software (malware). Fortunately, there are various ways to counteract these attacks and prevent them. One of these methods is developing a comprehensive malware dataset that researchers can utilize for malware analysis, detection, prediction, and prevention systems. This paper introduces a unique, up-to-date, labeled Android malware dataset (Maloid-DS) comprising a comprehensive set of malware families that reached 345 families with 47,971 malware samples. First, we intensely studied existing datasets utilized by previous research works. These datasets are limited in (a) the number of studied families, (b) the number of samples under each family, (c) the number of new malware samples, (d) the proper categorization of the malware families, (e) the accurate mapping of the sample with its corresponding malware family, (f) providing well structuring of the malware families and subfamilies, and (g) presenting a profound description of each family behavior. All these limitations were seriously tackled by introducing Maloid-DS. The process of creating Maloid is detailed in this paper. Moreover, several case studies are demonstrated in this paper to show the value of Maloid and how different types of analysis systems and AI-based detection and prediction solutions could utilize it. While the full potential of Maloid-DS in real-world scenarios is subject to ongoing research and practical application, it represents a substantial contribution to the cybersecurity community, offering a broad and detailed foundation for protecting Android devices against malware threats.

Ensuring flight safety for unmanned aerial vehicles (UAVs) is a critical concern, necessitating effective mathematical modeling for safety forecasting in both academic and industrial contexts. This study addresses this need by combining the capabilities of deep neural networks and grey analysis to create a comprehensive mathematical modeling approach focused on spatial information service (SIS). The paper introduces a novel spatial information-aware flight security forecasting model for UAVs, emphasizing the transformative impact of the new methodology. Traditionally, factors influencing flight safety are identified and formulated based on SIS chain technology, SIS management rules, SIS business processes, and spatial SIS chain verification. To address the challenges posed by significant data volatility and missing data in the sample dateset, a non-equally spaced GM (1,1) model with an approximate non-simultaneous exponential law series is developed for prediction. Subsequently, multiple influencing factors are encoded and input into a specific BP neural network structure. The paper concludes with simulation experiments to evaluate the proposed model. The results of the simulation analysis demonstrate that the integration of deep learning and grey analysis in the proposed model effectively recognizes flight security risks with high efficiency. This underscores the transformative potential of the new approach in enhancing UAV flight safety forecasting.

Traffic Classification (TC) systems are designed to identify the applications generating network traffic. Recent advancements in TC leverage Deep Learning (DL) techniques, surpassing traditional methods in complex scenarios, including those with encrypted traffic. Notably, state-of-the-art DL-based TC systems have been developed for wireless networks using Physical Layer (L1) packets. This approach overcomes the common limitation in TC research that assumes traffic flows within a wired network under a single network management domain. Despite their benefits, DL-based TC systems often demand significant computational resources, typically available only in cloud environments. Consequently, deploying models at the edge is often infeasible due to their resource-intensive nature, given their original training and optimization for high-resource environments. The inherent challenge lies in adapting these systems for edge computing scenarios, including deployment at access points. In this paper, we propose a novel methodology that exploits expert knowledge in combination with recent advances in Multi-Task Learning (MTL) and Deep Neural Network (DNN) optimization to allow spectrum-based TC systems to run on constrained devices. This paper propose a well-defined and innovative methodology for resource-efficient, spectrum-based TC to address this issue, combining MTL with DNN optimization techniques. Performance evaluations on an NVIDIA Jetson TX2 demonstrate that our most optimized MTL model, handling four TC tasks, can reduce memory requirements by a factor of 2.65x and improve execution time by 3.6x compared to sequential execution of four Single-Task Learning (STL) models in a server-grade configuration, with minimal accuracy impact (less than a 0.5% drop) and energy efficiency of 0.97 millijoules per sample at inference. Compared to other edge platforms such as the Raspberry Pi model 3B+ (RPI3B+) with a low-power Artificial Intelligence (AI)-accelerator such as the Coral Tensor Processing Unit (TPU), the NVIDIA Jetson achieves a 12-fold improvement in energy efficiency with no impact on accuracy.These are the first available results to provide a benchmark for different performance metrics (memory, computing, energy) over heterogeneous constrained devices for this type of TC system.

Early detection of driver behavior is a pivotal aspect in enhancing road safety, focusing on identifying and mitigating risky driving patterns before they lead to accidents. The use of smartphone sensors for data acquisition marks a significant advancement in this field. It allows for continuous, real-time monitoring of driving patterns without the need for specialized equipment. In this study, we leverage a publicly available smartphone motion sensor dataset, utilizing accelerometer and gyroscope data from a Samsung Galaxy S21 to analyze driving behaviors classified as slow, normal, and aggressive. This research introduces a novel feature engineering technique named the RKnD (Random forest, K-nearest classifier, Decision tree) probabilistic feature engineering technique, which integrates three prominent machine learning (ML) models. This blend offers a robust analysis of driver behavior, leveraging the strengths of each algorithm. This paper emphasizes the importance of data balancing in machine learning, employing the Synthetic Minority Oversampling Technique (SMOTE) to enhance the reliability of the predictions. Furthermore, k-fold cross-validation is used to ensure the model’s consistency and accuracy across original features and the proposed RKnD probabilistic features of the data sets. By achieving such high accuracy, the study demonstrates the potential of smartphone-based systems to significantly improve road safety. This paper introduces a novel approach utilizing smartphone motion sensor data to detect driver behaviors with a remarkable accuracy rate of 99.63%. This research stands out for its application of machine learning techniques in a practical, accessible manner. This pioneering approach named RKnD feature engineering sets a new standard in the realm of smart transportation systems, opening avenues for further innovations in the field, and filling a gap in road safety analysis to avoid road accidents. Future research on RKnD should streamline its algorithm for real-time use, diversify datasets, integrate advanced Deep Learning for complex pattern detection, and undertake real-world testing to validate practicality and uncover challenges.

In view of the low segmentation accuracy for small-scale object and insufficient segmentation of local boundary for semantic segmentation methods based on Deep Learning, this paper proposes an image semantic segmentation approach based on attention mechanism and feature fusion. On the basis of ensuring the overall accuracy, the segmentation accuracy of small-scale object and local boundary is improved, and it meets the requirement of accurately segmenting the object in the complex background. Firstly, an image semantic segmentation model based on hybrid cascade and feature fusion is proposed, and the hybrid concatenation and multi-cores pooling methods are used to extract deeper semantic information. Then, a cross-stages fusion approach is designed to divide the backbone network of the encoder stage in the network and the improved Atrous Spatial Pyramid Pooling module into three stages to fully utilize the different semantic information of the shallow and deep layers. Thirdly, the attention mechanism is introduced into the hybrid cascade and feature fusion image semantic segmentation network model, and image semantic segmentation model based on cross-stages and attention mechanisms is explained. Self attention is added to channel attention enhances the connection between feature maps, and one-dimensional convolution in the spatial attention mechanism is used to increase the spatial receptive field. The final results on the public dataset PASCAL VOC2012 and SUIM show that MIoUs have reached 86.68% and 61.55% respectively, and it proved that the overall accuracy of the approach proposed in this paper is higher than other ones.

This work reviews the critical challenge of data scarcity in developing Transformer-based models for Electroencephalography (EEG)-based Brain-Computer Interfaces (BCIs), specifically focusing on Motor Imagery (MI) decoding. While EEG-BCIs hold immense promise for applications in communication, rehabilitation, and human-computer interaction, limited data availability hinders the use of advanced deep-learning models such as Transformers. In particular, this paper comprehensively analyzes three key strategies to address data scarcity: data augmentation, transfer learning, and the inherent attention mechanisms of Transformers. Data augmentation techniques artificially expand datasets, enhancing model generalizability by exposing them to a wider range of signal patterns. Transfer learning utilizes pre-trained models from related domains, leveraging their learned knowledge to overcome the limitations of small EEG datasets. By thoroughly reviewing current research and methodologies, this work underscores the importance of these strategies in overcoming data scarcity. It critically examines the limitations imposed by limited datasets and showcases potential solutions being developed to address these challenges. This comprehensive survey, focusing on the intersection of data scarcity and technological advancements, aims to provide a critical analysis of the current state-of-the-art in EEG-BCI development. By identifying research gaps and suggesting future directions, the paper encourages further exploration and innovation in this field. Ultimately, this work aims to contribute to the advancement of more accessible, efficient, and accurate EEG-BCI systems by addressing the fundamental challenge of data scarcity.

Deep learning has revolutionized object detection, with YOLO (You Only Look Once) leading in real-time accuracy. However, detecting moving objects in visual streams presents distinct challenges. This paper proposes a refined YOLOv8 object detection model, emphasizing motion-specific detections in varied visual contexts. Through tailored preprocessing and architectural adjustments, we heighten the model’s sensitivity to object movements. Rigorous testing against KITTI, LASIESTA, PESMOD, and MOCS benchmark datasets revealed that the modified YOLOv8 outperforms the state-of-the-art detection models, especially in environments with significant movement. Specifically, our model achieved an accuracy of 90%, a mean Average Precision (mAP) of 90%, and maintained a processing speed of 30 frames per second (FPS), with an Intersection over Union (IoU) score of 80%. This paper offers a detailed insight into object trajectories, proving invaluable in areas like security, traffic management, and film analysis where motion understanding is critical. As the importance of dynamic scene interpretation grows in artificial intelligence and computer vision, the proposed enhanced YOLOv8 detection model highlights the potential of specialized object detection and underscores the significance of our findings in the evolving field of object detection.

The real-time analysis of gas flares is one of the most challenging problems in the operation of various combustion-involving industries, such as oil and gas refineries. Despite the crucial role of gas flares in securing safe plant operation and lowering environmental pollution, they are among the least monitored components of petrochemical plants due to their harsh working environments. Remote sensing techniques are emerging as potential alternatives for conventional sampling-based techniques in visual inspection and performance analysis. This paper presents an in-depth review of significant achievements in Gas flares monitoring over past two decades and highlights ongoing challenges in Artificial Intelligence (AI)-enhanced remote inspection. By reading the content, both industry professionals and academic researchers can gain a comprehensive knowledge of improvements from the integration of AI and remote sensing, understanding the trend of Gas flares monitoring. The paper commences with an analysis of RGB camera-based methods, focusing on how their combination with cutting-edge machine learning and deep learning algorithms can significantly improve the detection, segmentation, and measurement of gas flare systems. It then explores the use of hyper-spectral imaging techniques, including infrared cameras and space-borne satellite sensors, underscoring their potential in remote monitoring and performance analysis. Additionally, the effectiveness of multi-view inspection methods is assessed, highlighting how these approaches enhance the monitoring capabilities. Finally, the paper identifies key research areas that require further attention. It also presents a clearer direction for future progress, emphasizing the importance of continuous research to foster advancements and facilitate broader commercial adoption.

Speech and voice disorders are highly prevalent and a significant concern, particularly among children. These disorders have a notable impact on individual’s personalities, academic performance and overall development. Early detection and intervention of these disorders surely help out and generate an effective outcome. Automatic detection of speech and voice disorders at an early stage plays a crucial role in identifying and addressing these communication challenges because of their efficiency over traditional diagnostic methods. Over time, numerous techniques were developed in the past, but some of them have become less efficient in the present scenario. Deep Learning has surfaced as a recent and significant advancement in detecting speech and voice disorders. This paper provides a systematic review of the utilization of deep learning techniques for the detection of speech and voice disorders. The review encompasses studies published from 2018 to 2023 exploring various architectures of deep learning models for capturing complex patterns in speech data. Each deep learning based speech sound disorder detection technique is discussed with critical appraisal and relevant benchmarks available for evaluation of results. This review holds significance for new researchers who are interested in exploring the field of automatic speech disorder detection as the paper concludes by discussing future directions and potential areas of improvement in automatic speech sound disorder detection using deep learning.

Rare event detection (RED) involves the identification and detection of events characterized by low frequency of occurrences, but of high importance or impact. This paper presents a Systematic Review (SR) of rare event detection across various modalities using Machine Learning (ML) and Deep Learning (DL) techniques. This review comprehensively outlines techniques and methods best suited for rare event detection across various modalities, while also highlighting future research prospects. To the extent of our knowledge, this paper is a pioneering SR dedicated to exploring this specific research domain. This SR identifies the employed methods and techniques, the datasets utilized, and the effectiveness of these methods in detecting rare events. Four modalities concerning RED are reviewed in this SR: video, sound, image, and time series. The corresponding performances for the different ML and DL techniques for RED are discussed comprehensively, together with the associated RED challenges and limitations as well as the directions for future research are highlighted. This SR aims to offer a comprehensive overview of the existing methods in RED, serving as a valuable resource for researchers and practitioners working in the respective field.

This paper aims to develop a defect-background separated generative adversarial network (GAN) using deep learning and GAN to enhance the accuracy of battery exterior defect inspection. In actual battery production lines, the occurrence rates of defects vary by defect type, making it challenging to create a large, uniform defect dataset due to the time required for defect acquisition. This leads to a reduction in the accuracy of battery exterior defect inspection. To construct a large, uniform defect dataset, this paper proposes a defect-background separated GAN based on the principles of GANs. The defect-background separated GAN performs effective defect and background separation learning by referring to the segmentation labeling of defects. Through dataset augmentation using the defect-background separated GAN, the performance quality of newly generated synthetic defect images has improved, and accuracy in battery exterior defect inspection can be enhanced through extensive dataset training. Experimental results show the lowest Fréchet inception distance score among various other methods for the battery exterior defect dataset, and it generates clear synthetic defects perceptible to the human eye. By training defect segmentation on this large, uniform defect dataset, an accuracy of 96.1% and an intersection over union value of 0.71 were achieved. Ultimately, applying this defect inspection network to the actual production line demonstrated a 72% improvement in time efficiency. This demonstrates the stability and robustness of the large, uniform defect dataset generated through the defect-background separated GAN.

It is important in production to achieve accurate counting and density estimation of high-density culture fry under the environmental conditions of aquaculture scenarios in an efficient and accurate manner. However, none of the current methods for fry counting works well under the high-density and high-overlap conditions of real aquaculture scenarios. Therefore, in this paper, we propose a high-density farming fry monitoring network model, Super-Resolution GAN Density Estimate Attention Network (SGDAN), which incorporating an image enhancement algorithm and an attention mechanism, and we create a high-density farming fry dataset (HD-FryDataset) based on the environmental conditions of real aquaculture scenarios. The network model is designed to improve and optimize the targeted subnetworks for several key aspects of high-density fish fry monitoring work. Four subnetworks are included for image optimization, feature extraction, attention, and density map estimation. The experimental results show that the SGDAN network model achieved an average counting accuracy of 97.57% on the high-density culture fry dataset, which was 8.23% and 2.06% higher than those of MCNN and CSRNet, respectively. Additionally, the MAE and RMSE of the model were reduced by 71.9% and 67.3% and by 34.3% and 33.2% compared with those of MCNN and CSRNet, respectively. The model proposed in this paper also has a better ability to generate predictive density maps. The density maps generated by SGDAN have values of the evaluation metrics PSNR and SSIM of 20.33 and 0.933, respectively, which are 3.31 and 0.037 and 2.63 and 0.031 higher than those of MCNN and CSRNet. In general, the network model proposed in this paper outperforms existing network models in two applications: accurate counting of fry and generation of density maps for high-density culture in aquaculture. It also provides a good solution for digitizing the number of fry and visualizing the density of high-density culture in intelligent aquaculture systems.

Vision-based smart agriculture is an important way to improve the efficiency of agricultural production. Labeling images for deep learning in complex field photos is a difficult task. In this paper, an image augmentation method that can help reduce the workload of image labeling based on synthetic images is proposed. The synthetic images consisted of three parts: crop, weed, and soil. The crop and weeds were obtained automatically by Excess Green (ExG) and minimum error threshold segmentation. The data augmentation method was tested on image classification, object detection, and semantic segmentation tasks by Resnet, YOLOV5, and DeeplabV3. The accuracy of the classification model reached 0.99. The IoU of object detection and semantic segmentation were 0.98 and 0.96, respectively. The results showed that the method in this paper was acceptable despite slight overfitting. This method was proposed based on the characteristics of field images, and it was meant for reducing the workload of labeling images.

This paper provides a review of the most recent advances in artificial intelligence (AI) as applied to computational electromagnetics (CEM) to address challenges and unlock opportunities in power system applications. It is intended to provide readers and practitioners in electromagnetics (EM) and related applicable fields with valuable perspectives on the efficiency and capabilities of machine learning (ML) techniques used with CEM tools, offering unparalleled computational advantage. The discussion begins with an overview of traditional computational methods in EM, highlighting their strengths and limitations. The paper then delves into the integration of AI techniques, including ML, deep learning, and optimization algorithms, into CEM frameworks. Emphasis is placed on how AI enhances the accuracy and efficiency of EM simulations, enabling rapid analysis and optimization of power system components and configurations. Case studies and examples illustrate the successful application of AI-based CEM in solving practical challenges in electrical machine modeling, condition monitoring, and design optimizations in power systems. This paper conducts a comprehensive assessment of AI-based CEM techniques, critically evaluating their merits, addressing open issues, and examining the technical implementations within the context of power system applications.

The rapid advancement of Large Language Models (LLMs) has significantly enhanced the capabilities of text generators. With the potential for misuse escalating, the importance of discerning whether texts are human-authored or generated by LLMs has become paramount. Several preceding studies have ventured to address this challenge by employing binary classifiers to differentiate between human-written and LLM-generated text. Nevertheless, the reliability of these classifiers has been subject to question. Given that consequential decisions may hinge on the outcome of such classification, it is imperative that text source detection is of high caliber. In light of this, the present paper introduces DeepTextMark, a deep learning-driven text watermarking methodology devised for text source identification. By leveraging Word2Vec and Sentence Encoding for watermark insertion, alongside a transformer-based classifier for watermark detection, DeepTextMark epitomizes a blend of blindness, robustness, imperceptibility, and reliability. As elaborated within the paper, these attributes are crucial for universal text source detection, with a particular emphasis in this paper on text produced by LLMs. DeepTextMark offers a viable “add-on” solution to prevailing text generation frameworks, requiring no direct access or alterations to the underlying text generation mechanism. Experimental evaluations underscore the high imperceptibility, elevated detection accuracy, augmented robustness, reliability, and swift execution of DeepTextMark.

Environment perception using camera, radar, and/or lidar sensors has significantly improved in the last few years because of deep learning-based methods. However, a large group of these methods fall into the category of supervised learning, which requires a considerable amount of annotated data. Due to uncertainties in multi-sensor data, automating the data labeling process is extremely challenging; hence, it is performed manually to a large extent. Even though full automation of such a process is difficult, semi-automation can be a significant step to ease this process. However, the available work in this regard is still very limited; hence, in this paper, a novel semi-automatic annotation methodology is developed for labeling RGB camera images and 3D automotive radar point cloud data using a smart infrastructure-based sensor setup. This paper also describes a new method for 3D radar background subtraction to remove clutter and a new object category, GROUP, for radar-based object detection for closely located vulnerable road users. To validate the work, a dataset named INFRA-3DRC is created using this methodology, where 75 % of the labels are automatically generated. In addition, a radar cluster classifier and an image classifier are developed, trained, and tested on this dataset, achieving accuracy of 98.26% and 94.86%, respectively. The dataset and Python scripts are available at https://fraunhoferivi.github.io/INFRA-3DRC-Dataset/.