Search

Analysis of unintended compromising emissions from Video Display Units (VDUs) is an important topic in research communities. This paper examines the feasibility of recovering the information displayed on the monitor from reconstructed video frames. The study holds particular significance for our understanding of security vulnerabilities associated with the electromagnetic radiation of digital displays. Considering the amount of noise that reconstructed TEMPEST video frames have, the work in this paper focuses on two different approaches to de-noising images for efficient optical character recognition. First, an Adaptive Wiener Filter (AWF) with adaptive window size implemented in the spatial domain was tested, and then a Convolutional Neural Network (CNN) with an encoder–decoder structure that follows both classical auto-encoder model architecture and U-Net architecture (auto-encoder with skip connections). These two techniques resulted in an improvement of more than two times on the Structural Similarity Index Metric (SSIM) for AWF and up to four times for the SSIM for the Deep Learning (DL) approach. In addition, to validate the results, the possibility of text recovery from processed noisy frames was studied using a state-of-the-art Tesseract Optical Character Recognition (OCR) engine. The present work aims to bring to attention the security importance of this topic and the non-negligible character of VDU information leakages.

With the increasing number of households owning pets, the importance of sensor data for recognizing pet behavior has grown significantly. However, challenges arise due to the costs and reliability issues associated with data collection. This paper proposes a method for classifying pet behavior using cleaned meta pseudo labels to overcome these issues. The data for this study were collected using wearable devices equipped with accelerometers, gyroscopes, and magnetometers, and pet behaviors were classified into five categories. Utilizing this data, we analyzed the impact of the quantity of labeled data on accuracy and further enhanced the learning process by integrating an additional Distance Loss. This method effectively improves the learning process by removing noise from unlabeled data. Experimental results demonstrated that while the conventional supervised learning method achieved an accuracy of 82.9%, the existing meta pseudo labels method showed an accuracy of 86.2%, and the cleaned meta pseudo labels method proposed in this study surpassed these with an accuracy of 88.3%. These results hold significant implications for the development of pet monitoring systems, and the approach of this paper provides an effective solution for recognizing and classifying pet behavior in environments with insufficient labels.

Delamination represents one of the most significant and dangerous damages in composite plates. Recently, many papers have presented the capability of structural health monitoring (SHM) techniques for the investigation of structural delamination with various shapes and thickness depths. However, few studies have been conducted regarding the utilization of convolutional neural network (CNN) methods for automating the non-destructive testing (NDT) techniques database to identify the delamination size and depth. In this paper, an automated system qualified for distinguishing between pristine and damaged structures and classifying three classes of delamination with various depths is presented. This system includes a proposed CNN model and the Lamb wave technique. In this work, a unidirectional composite plate with three samples of delamination inserted at different depths was prepared for numerical and experimental investigations. In the numerical part, the guided wave propagation and interaction with three samples of delamination were studied to observe how the delamination depth can affect the scattered and trapped waves over the delamination region. This numerical study was validated experimentally using an efficient ultrasonic guided waves technique. This technique involved piezoelectric wafer active sensors (PWASs) and a scanning laser Doppler vibrometer (SLDV). Both numerical and experimental studies demonstrate that the delamination depth has a direct effect on the trapped waves’ energy and distribution. Three different datasets were collected from the numerical and experimental studies, involving the numerical wavefield image dataset, experimental wavefield image dataset, and experimental wavenumber spectrum image dataset. These three datasets were used independently with the proposed CNN model to develop a system that can automatically classify four classes (pristine class and three different delamination classes). The results of all three datasets show the capability of the proposed CNN model for predicting the delamination depth with high accuracy. The proposed CNN model results of the three different datasets were validated using the GoogLeNet CNN. The results of both methods show an excellent agreement. The results proved the capability of the wavefield image and wavenumber spectrum datasets to be used as input data to the CNN for the detection of delamination depth.

The main purpose of this paper is to provide information on how to create a convolutional neural network (CNN) for extracting features from EEG signals. Our task was to understand the primary aspects of creating and fine-tuning CNNs for various application scenarios. We considered the characteristics of EEG signals, coupled with an exploration of various signal processing and data preparation techniques. These techniques include noise reduction, filtering, encoding, decoding, and dimension reduction, among others. In addition, we conduct an in-depth analysis of well-known CNN architectures, categorizing them into four distinct groups: standard implementation, recurrent convolutional, decoder architecture, and combined architecture. This paper further offers a comprehensive evaluation of these architectures, covering accuracy metrics, hyperparameters, and an appendix that contains a table outlining the parameters of commonly used CNN architectures for feature extraction from EEG signals.

In this paper, research was conducted on anomaly detection of wheel flats. In the railway sector, conducting tests with actual railway vehicles is challenging due to safety concerns for passengers and maintenance issues as it is a public industry. Therefore, dynamics software was utilized. Next, STFT (short-time Fourier transform) was performed to create spectrogram images. In the case of railway vehicles, control, monitoring, and communication are performed through TCMS, but complex analysis and data processing are difficult because there are no devices such as GPUs. Furthermore, there are memory limitations. Therefore, in this paper, the relatively lightweight models LeNet-5, ResNet-20, and MobileNet-V3 were selected for deep learning experiments. At this time, the LeNet-5 and MobileNet-V3 models were modified from the basic architecture. Since railway vehicles are given preventive maintenance, it is difficult to obtain fault data. Therefore, semi-supervised learning was also performed. At this time, the Deep One Class Classification paper was referenced. The evaluation results indicated that the modified LeNet-5 and MobileNet-V3 models achieved approximately 97% and 96% accuracy, respectively. At this point, the LeNet-5 model showed a training time of 12 min faster than the MobileNet-V3 model. In addition, the semi-supervised learning results showed a significant outcome of approximately 94% accuracy when considering the railway maintenance environment. In conclusion, considering the railway vehicle maintenance environment and device specifications, it was inferred that the relatively simple and lightweight LeNet-5 model can be effectively utilized while using small images.

Negative emotions of drivers may lead to some dangerous driving behaviors, which in turn lead to serious traffic accidents. However, most of the current studies on driver emotions use a single modality, such as EEG, eye trackers, and driving data. In complex situations, a single modality may not be able to fully consider a driver’s complete emotional characteristics and provides poor robustness. In recent years, some studies have used multimodal thinking to monitor single emotions such as driver fatigue and anger, but in actual driving environments, negative emotions such as sadness, anger, fear, and fatigue all have a significant impact on driving safety. However, there are very few research cases using multimodal data to accurately predict drivers’ comprehensive emotions. Therefore, based on the multi-modal idea, this paper aims to improve drivers’ comprehensive emotion recognition. By combining the three modalities of a driver’s voice, facial image, and video sequence, the six classification tasks of drivers’ emotions are performed as follows: sadness, anger, fear, fatigue, happiness, and emotional neutrality. In order to accurately identify drivers’ negative emotions to improve driving safety, this paper proposes a multi-modal fusion framework based on the CNN + Bi-LSTM + HAM to identify driver emotions. The framework fuses feature vectors of driver audio, facial expressions, and video sequences for comprehensive driver emotion recognition. Experiments have proved the effectiveness of the multi-modal data proposed in this paper for driver emotion recognition, and its recognition accuracy has reached 85.52%. At the same time, the validity of this method is verified by comparing experiments and evaluation indicators such as accuracy and F1 score.

Recently, convolutional neural networks (CNNs) have been widely used in image denoising, and their performance has been enhanced through residual learning. However, previous research mostly focused on optimizing the network architecture of CNNs, ignoring the limitations of the commonly used residual learning. This paper identifies two of its limitations, which are the neglect of image information and the lack of effective consideration of image self-similarity. To solve these limitations, this paper proposes a compositional denoising network (CDN), which contains two sub-paths, the image information path (IIP) and the noise estimation path (NEP), respectively. IIP is trained via an image-to-image method to extract image information. For NEP, it utilizes image self-similarity from the perspective of training. This similarity-based training method constrains NEP to output similar estimated noise distributions for different image patches with a specific kind of noise. Finally, image information and noise distribution information are comprehensively considered for image denoising. Experimental results indicate that CDN outperforms other CNN-based methods in both synthetic and real-world image denoising, achieving state-of-the-art performance.

Tool wear condition monitoring is an important component of mechanical processing automation, and accurately identifying the wear status of tools can improve processing quality and production efficiency. This paper studied a new deep learning model, to identify the wear status of tools. The force signal was transformed into a two-dimensional image using continuous wavelet transform (CWT), short-time Fourier transform (STFT), and Gramian angular summation field (GASF) methods. The generated images were then fed into the proposed convolutional neural network (CNN) model for further analysis. The calculation results show that the accuracy of tool wear state recognition proposed in this paper was above 90%, which was higher than the accuracy of AlexNet, ResNet, and other models. The accuracy of the images generated using the CWT method and identified with the CNN model was the highest, which is attributed to the fact that the CWT method can extract local features of an image and is less affected by noise. Comparing the precision and recall values of the model, it was verified that the image obtained by the CWT method had the highest accuracy in identifying tool wear state. These results demonstrate the potential advantages of using a force signal transformed into a two-dimensional image for tool wear state recognition and of applying CNN models in this area. They also indicate the wide application prospects of this method in industrial production.

Detecting and classifying vehicles as objects from images and videos is challenging in appearance-based representation, yet plays a significant role in the substantial real-time applications of Intelligent Transportation Systems (ITSs). The rapid development of Deep Learning (DL) has resulted in the computer-vision community demanding efficient, robust, and outstanding services to be built in various fields. This paper covers a wide range of vehicle detection and classification approaches and the application of these in estimating traffic density, real-time targets, toll management and other areas using DL architectures. Moreover, the paper also presents a detailed analysis of DL techniques, benchmark datasets, and preliminaries. A survey of some vital detection and classification applications, namely, vehicle detection and classification and performance, is conducted, with a detailed investigation of the challenges faced. The paper also addresses the promising technological advancements of the last few years.

The survival rate for sudden cardiac arrest (SCA) is low, and patients with long-term risks of SCA are not adequately alerted. Understanding SCA’s characteristics will be key to developing preventive strategies. Many lives could be saved if SCA’s early onset could be detected or predicted. Monitoring heart signals continuously is essential for diagnosing sporadic cardiac dysfunction. An electrocardiogram (ECG) can be used to continuously monitor heart function without having to go to the hospital. A zeolite-based dry electrode can provide safe on-skin ECG acquisition while the subject is out-of-hospital and facilitate long-term monitoring. To the ECG signal, a low-power 1 μW read-out circuit was designed and implemented in our prior work. However, having long-term ECG monitoring outside the hospital, i.e., high battery life, and low power consumption while transmission and reception of ECG signal are crucial. This paper proposes a prototype with a 10-bit resolution ADC and nRF24L01 transceivers placed 5 m apart. The system uses the 2.4 GHz worldwide ISM frequency band with GFSK modulation to wirelessly transmit digitized ECG bits at 250 kbps data rate to a physician’s computer (or similar) for continuous monitoring of ECG signals; the power consumption is only 11.2 mW and 4.62 mW during transmission and reception, respectively, with a low bit error rate of ≤0.1%. Additionally, a subject-wise cross-validated, three-fold, optimized convolutional neural network (CNN) model using the Physionet-SCA dataset was implemented on NVIDIA Jetson to identify the irregular heartbeats yielding an accuracy of 89% with a run time of 5.31 s. Normal beat classification has an F1 score of 0.94 and a ROC score of 0.886. Thus, this paper integrates the ECG acquisition and processing unit with low-power wireless transmission and CNN model to detect irregular heartbeats.

Since the 20th century, a rapid process of motorization has begun. The main goal of researchers, engineers and technology companies is to increase the safety and optimality of the movement of vehicles, as well as to reduce the environmental damage caused by the automotive industry. The difficulty of managing traffic flows is that cars are driven by a person and their behavior, even in similar situations, is different and difficult to predict. To solve this problem, ground-based unmanned vehicles are increasingly being developed and implemented; however, like any other intelligent system, it is necessary to train different road scenarios. Currently, an engineer is driving an unmanned vehicle for training and thousands of kilometers are being driven for training. Of course, this approach to training unmanned vehicles is very long, and it is impossible to reproduce all the scenarios that can be found in real operations on a real road. Based on this, we offer a simulator of a realistic urban environment which allows you to reduce the training time and allows you to generate all kinds of events. To implement such a simulator, it is necessary to develop a method that would allow recreating a realistic world in one passage with cameras (monocular) installed on board the vehicle. Based on this, the purpose of this work is to develop an intelligent vehicle recognition system using convolutional neural networks, which allows you to create mesh objects for further placement in the simulator. It is important to note that the resulting objects should be optimal in size so as not to overload the system, since a large number of road infrastructure objects are stored there. Also, neural complexity should not be excessive. In this paper, the general concept and classification of convolutional neural networks are given, which allow solving the problem of recognizing 3D objects in images. Based on the analysis, the existing neural network architectures do not solve the problems mentioned above. In this connection, the authors first of all carried out the design of the system according to the methodology of modeling business processes, and also modified and developed the architecture of the neural network, which allows classifying objects with sufficient accuracy, obtaining optimized mesh objects and reducing computational complexity. The methods proposed in this paper are used in a simulator of a realistic urban environment, which reduces the time and computational costs when training unmanned transport systems.

This paper explores the utilization of smart device sensors for the purpose of vehicle recognition. Currently a ubiquitous aspect of people’s lives, smart devices can conveniently record details about walking, biking, jogging, and stepping, including physiological data, via often built-in phone activity recognition processes. This paper examines research on intelligent transportation systems to uncover how smart device sensor data may be used for vehicle recognition research, and fit within its growing body of literature. Here, we use the accelerometer and gyroscope, which can be commonly found in a smart phone, to detect the class of a vehicle. We collected data from cars, buses, trains, and bikes using a smartphone, and we designed a 1D CNN model leveraging the residual connection for vehicle recognition. The model achieved more than 98% accuracy in prediction. Moreover, we also provide future research directions based on our study.

Bird strikes are a substantial aviation safety issue that can result in serious harm to aircraft components and even passenger deaths. In response to this increased tendency, the implementation of new and more efficient detection and prevention technologies becomes urgent. The paper presents a novel deep learning model which is developed to detect and alleviate bird strike issues in airport conditions boosting aircraft safety. Based on an extensive database of bird images having different species and flight patterns, the research adopts sophisticated image augmentation techniques which generate multiple scenarios of aircraft operation ensuring that the model is robust under different conditions. The methodology evolved around the building of a spatiotemporal convolutional neural network which employs spatial attention structures together with dynamic temporal processing to precisely recognize flying birds. One of the most important features of this research is the architecture of its dual-focus model which consists of two components, the attention-based temporal analysis network and the convolutional neural network with spatial awareness. The model’s architecture can identify specific features nested in a crowded and shifting backdrop, thereby lowering false positives and improving detection accuracy. The mechanisms of attention of this model itself enhance the model’s focus by identifying vital features of bird flight patterns that are crucial. The results are that the proposed model achieves better performance in terms of accuracy and real time responses than the existing bird detection systems. The ablation study demonstrates the indispensable roles of each component, confirming their synergistic effect on improving detection performance. The research substantiates the model’s applicability as a part of airport bird strike surveillance system, providing an alternative to the prevention strategy. This work benefits from the unique deep learning feature application, which leads to a large-scale and reliable tool for dealing with the bird strike problem.

Using a sensor in variable lighting conditions, especially very low-light conditions, requires the application of image enhancement followed by denoising to retrieve correct information. The limits of such a process are explored in the present paper, with the objective of preserving the quality of enhanced images. The LIP (Logarithmic Image Processing) framework was initially created to process images acquired in transmission. The compatibility of this framework with the human visual system makes possible its application to images acquired in reflection. Previous works have established the ability of the LIP laws to perform a precise simulation of exposure time variation. Such a simulation permits the enhancement of low-light images, but a denoising step is required, realized by using a CNN (Convolutional Neural Network). A main contribution of the paper consists of using rigorous tools (metrics) to estimate the enhancement reliability in terms of noise reduction, visual image quality, and color preservation. Thanks to these tools, it has been established that the standard exposure time can be significantly reduced, which considerably enlarges the use of a given sensor. Moreover, the contribution of the LIP enhancement and denoising step are evaluated separately.

The fingerprint is one of the leading biometric modalities that is used worldwide for authenticating the identity of persons. Over time, a lot of research has been conducted to develop automatic fingerprint verification techniques. However, due to different authentication needs, the use of different sensors and the fingerprint verification systems encounter cross-sensor matching or sensor interoperability challenges, where different sensors are used for the enrollment and query phases. The challenge is to develop an efficient, robust and automatic system for cross-sensor matching. This paper proposes a new cross-matching system (SiameseFinger) using the Siamese network that takes the features extracted using the Gabor-HoG descriptor. The proposed Siamese network is trained using adversarial learning. The SiameseFinger was evaluated on two benchmark public datasets FingerPass and MOLF. The results of the experiments presented in this paper indicate that SiameseFinger achieves a comparable performance with that of the state-of-the-art methods.

In this paper, an innovative cyclic noise reduction method and an improved CAPON algorithm (also the called minimum variance distortionless response (MVDR) algorithm) are proposed to improve the accuracy and reliability of DOA (direction of arrival) estimation. By processing the eigenvalues obtained from the covariance matrix of the received signal, the signal-to-noise ratio (SNR) can be increased by up to 5 dB by the cyclic noise reduction method, which will improve the DOA estimation accuracy. The improved CAPON algorithm has a convolution neural network (CNN) structure, whose input is the processed covariance matrix of the received signal, and the CAPON spectral value is used as the training label to obtain the estimated spatial spectrum. It retains the advantages of the CAPON algorithm, which can achieve blind source estimation, and simulations show that the proposed algorithm exhibits a better performance than the traditional algorithm in conditions of various SNRs and snapshot numbers. The simulation results show that, based on a certain SNR, the root mean square error (RMSE) of the improved CAPON algorithm can be reduced from 0.86° to 0.8° compared to traditional algorithms, and the angle estimation error can be decreased by up to about 0.3°. With the help of the cyclic noise reduction method, the angle estimation error decreases from 0.04° to 0.02°.

Deaf and hard-of-hearing people mainly communicate using sign language, which is a set of signs made using hand gestures combined with facial expressions to make meaningful and complete sentences. The problem that faces deaf and hard-of-hearing people is the lack of automatic tools that translate sign languages into written or spoken text, which has led to a communication gap between them and their communities. Most state-of-the-art vision-based sign language recognition approaches focus on translating non-Arabic sign languages, with few targeting the Arabic Sign Language (ArSL) and even fewer targeting the Saudi Sign Language (SSL). This paper proposes a mobile application that helps deaf and hard-of-hearing people in Saudi Arabia to communicate efficiently with their communities. The prototype is an Android-based mobile application that applies deep learning techniques to translate isolated SSL to text and audio and includes unique features that are not available in other related applications targeting ArSL. The proposed approach, when evaluated on a comprehensive dataset, has demonstrated its effectiveness by outperforming several state-of-the-art approaches and producing results that are comparable to these approaches. Moreover, testing the prototype on several deaf and hard-of-hearing users, in addition to hearing users, proved its usefulness. In the future, we aim to improve the accuracy of the model and enrich the application with more features.

Insect monitoring methods are typically very time-consuming and involve substantial investment in species identification following manual trapping in the field. Insect traps are often only serviced weekly, resulting in low temporal resolution of the monitoring data, which hampers the ecological interpretation. This paper presents a portable computer vision system capable of attracting and detecting live insects. More specifically, the paper proposes detection and classification of species by recording images of live individuals attracted to a light trap. An Automated Moth Trap (AMT) with multiple light sources and a camera was designed to attract and monitor live insects during twilight and night hours. A computer vision algorithm referred to as Moth Classification and Counting (MCC), based on deep learning analysis of the captured images, tracked and counted the number of insects and identified moth species. Observations over 48 nights resulted in the capture of more than 250,000 images with an average of 5675 images per night. A customized convolutional neural network was trained on 2000 labeled images of live moths represented by eight different classes, achieving a high validation F1-score of 0.93. The algorithm measured an average classification and tracking F1-score of 0.71 and a tracking detection rate of 0.79. Overall, the proposed computer vision system and algorithm showed promising results as a low-cost solution for non-destructive and automatic monitoring of moths.

A thorough analysis of sports is becoming increasingly important during the training process of badminton players at both the recreational and professional level. Nowadays, game situations are usually filmed and reviewed afterwards in order to analyze the game situation, but these video set-ups tend to be difficult to analyze, expensive, and intrusive to set up. In contrast, we classified badminton movements using off-the-shelf accelerometer and gyroscope data. To this end, we organized a data capturing campaign and designed a novel neural network using different frame sizes as input. This paper shows that with only accelerometer data, our novel convolutional neural network is able to distinguish nine activities with 86% precision when using a sampling frequency of 50 Hz. Adding the gyroscope data causes an increase of up to 99% precision, as compared to, respectively, 79% and 88% when using a traditional convolutional neural network. In addition, our paper analyses the impact of different sensor placement options and discusses the impact of different sampling frequenciess of the sensors. As such, our approach provides a low cost solution that is easy to use and can collect useful information for the analysis of a badminton game.

In recent years, computer vision has witnessed remarkable advancements in image classification, specifically in the domains of fully convolutional neural networks (FCNs) and self-attention mechanisms. Nevertheless, both approaches exhibit certain limitations. FCNs tend to prioritize local information, potentially overlooking crucial global contexts, whereas self-attention mechanisms are computationally intensive despite their adaptability. In order to surmount these challenges, this paper proposes cross-and-diagonal networks (CDNet), innovative network architecture that adeptly captures global information in images while preserving local details in a more computationally efficient manner. CDNet achieves this by establishing long-range relationships between pixels within an image, enabling the indirect acquisition of contextual information. This inventive indirect self-attention mechanism significantly enhances the network’s capacity. In CDNet, a new attention mechanism named “cross and diagonal attention” is proposed. This mechanism adopts an indirect approach by integrating two distinct components, cross attention and diagonal attention. By computing attention in different directions, specifically vertical and diagonal, CDNet effectively establishes remote dependencies among pixels, resulting in improved performance in image classification tasks. Experimental results highlight several advantages of CDNet. Firstly, it introduces an indirect self-attention mechanism that can be effortlessly integrated as a module into any convolutional neural network (CNN). Additionally, the computational cost of the self-attention mechanism has been effectively reduced, resulting in improved overall computational efficiency. Lastly, CDNet attains state-of-the-art performance on three benchmark datasets for similar types of image classification networks. In essence, CDNet addresses the constraints of conventional approaches and provides an efficient and effective solution for capturing global context in image classification tasks.

The network intrusion detection system (NIDS) plays a crucial role as a security measure in addressing the increasing number of network threats. The majority of current research relies on feature-ready datasets that heavily depend on feature engineering. Conversely, the increasing complexity of network traffic and the ongoing evolution of attack techniques lead to a diminishing distinction between benign and malicious network behaviors. In this paper, we propose a novel end-to-end intrusion detection framework based on a contrastive learning approach. We design a hierarchical Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU) model to facilitate the automated extraction of spatiotemporal features from raw traffic data. The integration of contrastive learning amplifies the distinction between benign and malicious network traffic in the representation space. The proposed method exhibits enhanced detection capabilities for unknown attacks in comparison to the approaches trained using the cross-entropy loss function. Experiments are carried out on the public datasets CIC-IDS2017 and CSE-CIC-IDS2018, demonstrating that our method can attain a detection accuracy of 99.9% for known attacks, thus achieving state-of-the-art performance. For unknown attacks, a weighted recall rate of 95% can be achieved.

In this paper, we present a deep learning approach for identifying current intensity and frequency. The reconstruction is based on measurements of the magnetic field generated by the current flowing in a conductor. Magnetic field data are collected using a magnetic probe capable of generating a spectrogram, representing the spectrum of frequencies of the magnetic field over time. These spectrograms are saved as images characterized by color density proportional to the induction field value at a given frequency. The proposed deep learning approach utilizes a convolutional neural network (CNN) with the spectrogram image as input and the current or frequency value as output. One advantage of this approach is that current estimation is achieved contactless, using a simple magnetic field probe positioned close to the conductor.

Organizations managing high-performance computing systems face a multitude of challenges, including overarching concerns such as overall energy consumption, microprocessor clock frequency limitations, and the escalating costs associated with chip production. Evidently, processor speeds have plateaued over the last decade, persisting within the range of 2 GHz to 5 GHz. Scholars assert that brain-inspired computing holds substantial promise for mitigating these challenges. The spiking neural network (SNN) particularly stands out for its commendable power efficiency when juxtaposed with conventional design paradigms. Nevertheless, our scrutiny has brought to light several pivotal challenges impeding the seamless implementation of large-scale neural networks (NNs) on silicon. These challenges encompass the absence of automated tools, the need for multifaceted domain expertise, and the inadequacy of existing algorithms to efficiently partition and place extensive SNN computations onto hardware infrastructure. In this paper, we posit the development of an automated tool flow capable of transmuting any NN into an SNN. This undertaking involves the creation of a novel graph-partitioning algorithm designed to strategically place SNNs on a network-on-chip (NoC), thereby paving the way for future energy-efficient and high-performance computing paradigms. The presented methodology showcases its effectiveness by successfully transforming ANN architectures into SNNs with a marginal average error penalty of merely 2.65%. The proposed graph-partitioning algorithm enables a 14.22% decrease in inter-synaptic communication and an 87.58% reduction in intra-synaptic communication, on average, underscoring the effectiveness of the proposed algorithm in optimizing NN communication pathways. Compared to a baseline graph-partitioning algorithm, the proposed approach exhibits an average decrease of 79.74% in latency and a 14.67% reduction in energy consumption. Using existing NoC tools, the energy-latency product of SNN architectures is, on average, 82.71% lower than that of the baseline architectures.

The need for efficient video coding technology is more important than ever in the current scenario where video applications are increasing worldwide, and Internet of Things (IoT) devices are becoming widespread. In this context, it is necessary to carefully review the recently completed MPEG-5 Essential Video Coding (EVC) standard because the EVC Baseline profile is customized to meet the specific requirements needed to process IoT video data in terms of low complexity. Nevertheless, the EVC Baseline profile has a notable disadvantage. Since it is a codec composed only of simple tools developed over 20 years, it tends to represent numerous coding artifacts. In particular, the presence of blocking artifacts at the block boundary is regarded as a critical issue that must be addressed. To address this, this paper proposes a post-filter using a block partitioning information-based Convolutional Neural Network (CNN). The proposed method in the experimental results objectively shows an approximately 0.57 dB for All-Intra (AI) and 0.37 dB for Low-Delay (LD) improvements in each configuration by the proposed method when compared to the pre-post-filter video, and the enhanced PSNR results in an overall bitrate reduction of 11.62% for AI and 10.91% for LD in the Luma and Chroma components, respectively. Due to the huge improvement in the PSNR, the proposed method significantly improved the visual quality subjectively, particularly in blocking artifacts at the coding block boundary.

This paper introduces a noise augmentation technique designed to enhance the robustness of state-of-the-art (SOTA) deep learning models against degraded image quality, a common challenge in long-term recording systems. Our method, demonstrated through the classification of digital holographic images, utilizes a novel approach to synthesize and apply random colored noise, addressing the typically encountered correlated noise patterns in such images. Empirical results show that our technique not only maintains classification accuracy in high-quality images but also significantly improves it when given noisy inputs without increasing the training time. This advancement demonstrates the potential of our approach for augmenting data for deep learning models to perform effectively in production under varied and suboptimal conditions.

Hospitals need to invest a lot of manpower to manually input the contents of medical invoices (nearly 300,000,000 medical invoices a year) into the medical system. In order to help the hospital save money and stabilize work efficiency, this paper designed a system to complete the complicated work using a Gaussian blur and smoothing–convolutional neural network combined with a recurrent neural network (GBS-CR) method. Gaussian blur and smoothing (GBS) is a novel preprocessing method that can fix the breakpoint font in medical invoices. The combination of convolutional neural network (CNN) and recurrent neural network (RNN) was used to raise the recognition rate of the breakpoint font in medical invoices. RNN was designed to be the semantic revision module. In the aspect of image preprocessing, Gaussian blur and smoothing were used to fix the breakpoint font. In the period of making the self-built dataset, a certain proportion of the breakpoint font (the font of breakpoint is 3, the original font is 7) was added, in this paper, so as to optimize the Alexnet–Adam–CNN (AA-CNN) model, which is more suitable for the recognition of the breakpoint font than the traditional CNN model. In terms of the identification methods, we not only adopted the optimized AA-CNN for identification, but also combined RNN to carry out the semantic revisions of the identified results of CNN, meanwhile further improving the recognition rate of the medical invoices. The experimental results show that compared with the state-of-art invoice recognition method, the method presented in this paper has an average increase of 10 to 15 percentage points in recognition rate.

The popularity and demand for high-quality date palm fruits (Phoenix dactylifera L.) have been growing, and their quality largely depends on the type of handling, storage, and processing methods. The current methods of geometric evaluation and classification of date palm fruits are characterised by high labour intensity and are usually performed mechanically, which may cause additional damage and reduce the quality and value of the product. Therefore, non-contact methods are being sought based on image analysis, with digital solutions controlling the evaluation and classification processes. The main objective of this paper is to develop an automatic classification model for varieties of date palm fruits using a convolutional neural network (CNN) based on two fundamental criteria, i.e., colour difference and evaluation of geometric parameters of dates. A CNN with a fixed architecture was built, marked as DateNET, consisting of a system of five alternating Conv2D, MaxPooling2D, and Dropout classes. The validation accuracy of the model presented in this study depended on the selection of classification criteria. It was 85.24% for fruit colour-based classification and 87.62% for the geometric parameters only; however, it increased considerably to 93.41% when both the colour and geometry of dates were considered.

Ego-vehicle state prediction represents a complex and challenging problem for self-driving and autonomous vehicles. Sensorial information and on-board cameras are used in perception-based solutions in order to understand the state of the vehicle and the surrounding traffic conditions. Monocular camera-based methods are becoming increasingly popular for driver assistance, with precise predictions of vehicle speed and emergency braking being important for road safety enhancement, especially in the prevention of speed-related accidents. In this research paper, we introduce the implementation of a convolutional neural network (CNN) model tailored for the prediction of vehicle velocity, braking events, and emergency braking, employing sequential image sequences and velocity data as inputs. The CNN model is trained on a dataset featuring sequences of 20 consecutive images and corresponding velocity values, all obtained from a moving vehicle navigating through road-traffic scenarios. The model’s primary objective is to predict the current vehicle speed, braking actions, and the occurrence of an emergency-brake situation using the information encoded in the preceding 20 frames. We subject our proposed model to an evaluation on a dataset using regression and classification metrics, and comparative analysis with existing published work based on recurrent neural networks (RNNs). Through our efforts to improve the prediction accuracy for velocity, braking behavior, and emergency-brake events, we make a substantial contribution to improving road safety and offer valuable insights for the development of perception-based techniques in the field of autonomous vehicles.

A hypoglossal nerve stimulator (HGNS) is an invasive device that is used to treat obstructive sleep apnea (OSA) through electrical stimulation. The conventional implantable HGNS device consists of a stimuli generator, a breathing sensor, and electrodes connected to the hypoglossal nerve via leads. However, this implant is bulky and causes significant trauma. In this paper, we propose a minimally invasive HGNS based on an electrocardiogram (ECG) sensor and wireless power transfer (WPT), consisting of a wearable breathing monitor and an implantable stimulator. The breathing external monitor utilizes an ECG sensor to identify abnormal breathing patterns associated with OSA with 88.68% accuracy, achieved through the utilization of a convolutional neural network (CNN) algorithm. With a skin thickness of 5 mm and a receiving coil diameter of 9 mm, the power conversion efficiency was measured as 31.8%. The implantable device, on the other hand, is composed of a front-end CMOS power management module (PMM), a binary-phase-shift-keying (BPSK)-based data demodulator, and a bipolar biphasic current stimuli generator. The PMM, with a silicon area of 0.06 mm2 (excluding PADs), demonstrated a power conversion efficiency of 77.5% when operating at a receiving frequency of 2 MHz. Furthermore, it offers three-voltage options (1.2 V, 1.8 V, and 3.1 V). Within the data receiver component, a low-power BPSK demodulator was ingeniously incorporated, consuming only 42 μW when supplied with a voltage of 0.7 V. The performance was achieved through the implementation of the self-biased phase-locked-loop (PLL) technique. The stimuli generator delivers biphasic constant currents, providing a 5 bit programmable range spanning from 0 to 2.4 mA. The functionality of the proposed ECG- and WPT-based HGNS was validated, representing a highly promising solution for the effective management of OSA, all while minimizing the trauma and space requirements.

Human activity recognition (HAR) using wearable sensors enables continuous monitoring for healthcare applications. However, the conventional centralised training of deep learning models on sensor data poses challenges related to privacy, communication costs, and on-device efficiency. This paper proposes a federated learning framework integrating spiking neural networks (SNNs) with long short-term memory (LSTM) networks for energy-efficient and privacy-preserving HAR. The hybrid spiking-LSTM (S-LSTM) model synergistically combines the event-driven efficiency of SNNs and the sequence modelling capability of LSTMs. The model is trained using surrogate gradient learning and backpropagation through time, enabling fully supervised end-to-end learning. Extensive evaluations of two public datasets demonstrate that the proposed approach outperforms LSTM, CNN, and S-CNN models in accuracy and energy efficiency. For instance, the proposed S-LSTM achieved an accuracy of 97.36% and 89.69% for indoor and outdoor scenarios, respectively. Furthermore, the results also showed a significant improvement in energy efficiency of 32.30%, compared to simple LSTM. Additionally, we highlight the significance of personalisation in HAR, where fine-tuning with local data enhances model accuracy by up to 9% for individual users.

In this paper, research was conducted on Deep Learning Wavefront Sensing (DLWS) neural networks using simulated atmospheric turbulence datasets, and a novel DLWS was proposed based on attention mechanisms and Convolutional Neural Networks (CNNs). The study encompassed both indoor experiments and kilometer-range laser transmission experiments employing DLWS. In terms of indoor experiments, data were collected and training was performed on the platform built by us. Subsequent comparative experiments with the Shack-Hartmann Wavefront Sensing (SHWS) method revealed that our DLWS model achieved accuracy on par with SHWS. For the kilometer-scale experiments, we directly applied the DLWS model obtained from the indoor platform, eliminating the need for new data collection or additional training. The DLWS predicts the wavefront from the beacon light PSF in real time and then uses it for aberration correction of the emitted laser. The results demonstrate a substantial improvement in the average peak intensity of the light spot at the target position after closed-loop correction, with a remarkable increase of 5.35 times compared to the open-loop configuration.

The escalating global water usage and the increasing strain on major cities due to water shortages highlights the critical need for efficient water management practices. In water-stressed regions worldwide, significant water wastage is primarily attributed to leakages, inefficient use, and aging infrastructure. Undetected water leakages in buildings’ pipelines contribute to the water waste problem. To address this issue, an effective water leak detection method is required. In this paper, we explore the application of edge computing in smart buildings to enhance water management. By integrating sensors and embedded Machine Learning models, known as TinyML, smart water management systems can collect real-time data, analyze it, and make accurate decisions for efficient water utilization. The transition to TinyML enables faster and more cost-effective local decision-making, reducing the dependence on centralized entities. In this work, we propose a solution that can be adapted for effective leakage detection in real-world scenarios with minimum human intervention using TinyML. We follow an approach that is similar to a typical machine learning lifecycle in production, spanning stages including data collection, training, hyperparameter tuning, offline evaluation and model optimization for on-device resource efficiency before deployment. In this work, we considered an existing water leakage acoustic dataset for polyvinyl chloride pipelines. To prepare the acoustic data for analysis, we performed preprocessing to transform it into scalograms. We devised a water leak detection method by applying transfer learning to five distinct Convolutional Neural Network (CNN) variants, which are namely EfficientNet, ResNet, AlexNet, MobileNet V1, and MobileNet V2. The CNN models were found to be able to detect leakages where a maximum testing accuracy, recall, precision, and F1 score of 97.45%, 98.57%, 96.70%, and 97.63%, respectively, were observed using the EfficientNet model. To enable seamless deployment on the Arduino Nano 33 BLE edge device, the EfficientNet model is compressed using quantization resulting in a low inference time of 1932 ms, a peak RAM usage of 255.3 kilobytes, and a flash usage requirement of merely 48.7 kilobytes.

This paper presents a simple yet effective method for improving the performance of zero-shot learning (ZSL). ZSL classifies instances of unseen classes, from which no training data is available, by utilizing the attributes of the classes. Conventional ZSL methods have equally dealt with all the available attributes, but this sometimes causes misclassification. This is because an attribute that is effective for classifying instances of one class is not always effective for another class. In this case, a metric of classifying the latter class can be undesirably influenced by the irrelevant attribute. This paper solves this problem by taking the importance of each attribute for each class into account when calculating the metric. In addition to the proposal of this new method, this paper also contributes by providing a dataset for pose classification based on wearable sensors, named HDPoseDS. It contains 22 classes of poses performed by 10 subjects with 31 IMU sensors across full body. To the best of our knowledge, it is the richest wearable-sensor dataset especially in terms of sensor density, and thus it is suitable for studying zero-shot pose/action recognition. The presented method was evaluated on HDPoseDS and outperformed relative improvement of 5.9% in comparison to the best baseline method.

Poor working environment leads to frequent failures of planetary gear trains. However, complex structure and variable transmission make the vibration signal strongly non-linear and non-stationary, which brings big problems to fault diagnosis. A method of planetary gear fault diagnosis via feature image extraction based on multi central frequencies and vibration signal frequency spectrum is proposed. The original vibration signal is decomposed by variational mode decomposition (VMD), and four components with narrow bands and independent central frequencies are decomposed. In order to retain the feature spectrum of the original vibration signal as far as possible, the corresponding feature bands are intercepted from the frequency spectrum of original vibration signal based on the central frequency of each component. Then, the feature images of fault signals are constructed as the inputs of the convolution neural network (CNN), and the parameters of the neural network are optimized by sample training. Finally, the optimized CNN is used to identify fault signals. The overall fault recognition rate is up to 98.75%. Compared with the feature bands extracted directly from the component spectrums, the extraction method of the feature bands proposed in this paper needs fewer iterations under the same network structure. The method of planetary gear fault diagnosis proposed in this paper is effective.

Internet of Things (IoT) devices within smart cities, require innovative detection methods. This paper addresses this critical challenge by introducing a deep learning-based approach for the detection of network traffic attacks in IoT ecosystems. Leveraging the Kaggle dataset, our model integrates Convolutional Neural Networks (CNNs) and Gated Recurrent Units (GRUs) to capture both spatial and sequential features in network traffic data. We trained and evaluated our model over ten epochs, achieving an impressive overall accuracy rate of 99%. The classification report reveals the model’s proficiency in distinguishing various attack categories, including ‘Normal’, ‘DoS’ (Denial of Service), ‘Probe’, ‘U2R’ (User to Root), and ‘Sybil’. Additionally, the confusion matrix offers valuable insights into the model’s performance across these attack types. In terms of overall accuracy, our model achieves an impressive accuracy rate of 99% across all attack categories. The weighted- average F1-score is also 99%, showcasing the model’s robust performance in classifying network traffic attacks in IoT devices for smart cities. This advanced architecture exhibits the potential to fortify IoT device security in the complex landscape of smart cities, effectively contributing to the safeguarding of critical infrastructure

Alzheimer’s disease (AD), a neuropsychiatric disorder, continually arises in the elderly. To date, no targeted medications have been developed for AD. Early and fast diagnosis of AD plays a pivotal role in identifying potential AD patients, enabling timely medical interventions, and mitigating disease progression. Computer-aided diagnosis (CAD) becomes possible with the burgeoning of deep learning. However, the existing CAD models for processing 3D Alzheimer’s disease images usually have the problems of slow convergence, disappearance of gradient, and falling into local optimum. This makes the training of 3D diagnosis models need a lot of time, and the accuracy is often poor. In this paper, a novel 3D aggregated residual network with accelerated mirror descent optimization is proposed for diagnosing AD. First, a novel unbiased subgradient accelerated mirror descent (SAMD) optimization algorithm is proposed to speed up diagnosis network training. By optimizing the nonlinear projection process, our proposed algorithm can avoid the occurrence of the local optimum in the non-Euclidean distance metric. The most notable aspect is that, to the best of our knowledge, this is the pioneering attempt to optimize the AD diagnosis training process by improving the optimization algorithm. Then, we provide a rigorous proof of the SAMD’s convergence, and the convergence of SAMD is better than any existing gradient descent algorithms. Finally, we use our proposed SAMD algorithm to train our proposed 3D aggregated residual network architecture (ARCNN). We employed the ADNI dataset to train ARCNN diagnostic models separately for the AD vs. NC task and the sMCI vs. pMCI task, followed by testing to evaluate the disease diagnostic outcomes. The results reveal that the accuracy can be improved in diagnosing AD, and the training speed can be accelerated. Our proposed method achieves 95.4% accuracy in AD diagnosis and 79.9% accuracy in MCI diagnosis; the best results contrasted with several state-of-the-art diagnosis methods. In addition, our proposed SAMD algorithm can save about 19% of the convergence time on average in the AD diagnosis model compared with the gradient descent algorithms, which is very momentous in clinic.

Medical image segmentation is crucial for medical image processing and the development of computer-aided diagnostics. In recent years, deep Convolutional Neural Networks (CNNs) have been widely adopted for medical image segmentation and have achieved significant success. UNet, which is based on CNNs, is the mainstream method used for medical image segmentation. However, its performance suffers owing to its inability to capture long-range dependencies. Transformers were initially designed for Natural Language Processing (NLP), and sequence-to-sequence applications have demonstrated the ability to capture long-range dependencies. However, their abilities to acquire local information are limited. Hybrid architectures of CNNs and Transformer, such as TransUNet, have been proposed to benefit from Transformer’s long-range dependencies and CNNs’ low-level details. Nevertheless, automatic medical image segmentation remains a challenging task due to factors such as blurred boundaries, the low-contrast tissue environment, and in the context of ultrasound, issues like speckle noise and attenuation. In this paper, we propose a new model that combines the strengths of both CNNs and Transformer, with network architectural improvements designed to enrich the feature representation captured by the skip connections and the decoder. To this end, we devised a new attention module called Three-Level Attention (TLA). This module is composed of an Attention Gate (AG), channel attention, and spatial normalization mechanism. The AG preserves structural information, whereas channel attention helps to model the interdependencies between channels. Spatial normalization employs the spatial coefficient of the Transformer to improve spatial attention akin to TransNorm. To further improve the skip connection and reduce the semantic gap, skip connections between the encoder and decoder were redesigned in a manner similar to that of the UNet++ dense connection. Moreover, deep supervision using a side-output channel was introduced, analogous to BASNet, which was originally used for saliency predictions. Two datasets from different modalities, a CT scan dataset and an ultrasound dataset, were used to evaluate the proposed UNet architecture. The experimental results showed that our model consistently improved the prediction performance of the UNet across different datasets.

Alzheimer’s disease (AD) is a neurodegenerative disease that can cause dementia and result in a severe reduction in brain function, inhibiting simple tasks, especially if no preventative care is taken. Over 1 in 9 Americans suffer from AD-induced dementia, and unpaid care for people with AD-related dementia is valued at USD 271.6 billion. Hence, various approaches have been developed for early AD diagnosis to prevent its further progression. In this paper, we first review other approaches that could be used for the early detection of AD. We then give an overview of our dataset and propose a deep convolutional neural network (CNN) architecture consisting of 7,866,819 parameters. This model comprises three different convolutional branches, each having a different length. Each branch is comprised of different kernel sizes. This model can predict whether a patient is non-demented, mild-demented, or moderately demented with a 99.05% three-class accuracy. In summary, the deep CNN model demonstrated exceptional accuracy in the early diagnosis of AD, offering a significant advancement in the field and the potential to improve patient care.

Cattle behavior classification technology holds a crucial position within the realm of smart cattle farming. Addressing the requisites of cattle behavior classification in the agricultural sector, this paper presents a novel cattle behavior classification network tailored for intricate environments. This network amalgamates the capabilities of CNN and Bi-LSTM. Initially, a data collection method is devised within an authentic farm setting, followed by the delineation of eight fundamental cattle behaviors. The foundational step involves utilizing VGG16 as the cornerstone of the CNN network, thereby extracting spatial feature vectors from each video data sequence. Subsequently, these features are channeled into a Bi-LSTM classification model, adept at unearthing semantic insights from temporal data in both directions. This process ensures precise recognition and categorization of cattle behaviors. To validate the model’s efficacy, ablation experiments, generalization effect assessments, and comparative analyses under consistent experimental conditions are performed. These investigations, involving module replacements within the classification model and comprehensive analysis of ablation experiments, affirm the model’s effectiveness. The self-constructed dataset about cattle is subjected to evaluation using cross-entropy loss, assessing the model’s generalization efficacy across diverse subjects and viewing perspectives. Classification performance accuracy is quantified through the application of a confusion matrix. Furthermore, a set of comparison experiments is conducted, involving three pertinent deep learning models: MASK-RCNN, CNN-LSTM, and EfficientNet-LSTM. The outcomes of these experiments unequivocally substantiate the superiority of the proposed model. Empirical results underscore the CNN-Bi-LSTM model’s commendable performance metrics: achieving 94.3% accuracy, 94.2% precision, and 93.4% recall while navigating challenges such as varying light conditions, occlusions, and environmental influences. The objective of this study is to employ a fusion of CNN and Bi-LSTM to autonomously extract features from multimodal data, thereby addressing the challenge of classifying cattle behaviors within intricate scenes. By surpassing the constraints imposed by conventional methodologies and the analysis of single-sensor data, this approach seeks to enhance the precision and generalizability of cattle behavior classification. The consequential practical, economic, and societal implications for the agricultural sector are of considerable significance.

Crowd counting, as a basic computer vision task, plays an important role in many fields such as video surveillance, accident prediction, public security, and intelligent transportation. At present, crowd counting tasks face various challenges. Firstly, due to the diversity of crowd distribution and increasing population density, there is a phenomenon of large-scale crowd aggregation in public places, sports stadiums, and stations, resulting in very serious occlusion. Secondly, when annotating large-scale datasets, positioning errors can also easily affect training results. In addition, the size of human head targets in dense images is not consistent, making it difficult to identify both near and far targets using only one network simultaneously. The existing crowd counting methods mainly use density plot regression methods. However, this framework does not distinguish the features between distant and near targets and cannot adaptively respond to scale changes. Therefore, the detection performance in areas with sparse population distribution is not good. To solve such problems, we propose an adaptive multi-scale far and near distance network based on the convolutional neural network (CNN) framework for counting dense populations and achieving a good balance between accuracy, inference speed, and performance. However, on the feature level, in order to enable the model to distinguish the differences between near and far features, we use stacked convolution layers to deepen the depth of the network, allocate different receptive fields according to the distance between the target and the camera, and fuse the features between nearby targets to enhance the feature extraction ability of pedestrians under nearby targets. Secondly, depth information is used to distinguish distant and near targets of different scales and the original image is cut into four different patches to perform pixel-level adaptive modeling on the population. In addition, we add density normalized average precision (nAP) indicators to analyze the accuracy of our method in spatial positioning. This paper validates the effectiveness of NF-Net on three challenging benchmarks in Shanghai Tech Part A and B, UCF_ CC_50, and UCF-QNRF datasets. Compared with SOTA, it has more significant performance in various scenarios. In the UCF-QNRF dataset, it is further validated that our method effectively solves the interference of complex backgrounds.

High-efficiency video coding (HEVC/H.265) is one of the most widely used video coding standards. HEVC introduces a quad-tree coding unit (CU) partition structure to improve video compression efficiency. The determination of the optimal CU partition is achieved through the brute-force search rate-distortion optimization method, which may result in high encoding complexity and hardware implementation challenges. To address this problem, this paper proposes a method that combines convolutional neural networks (CNN) with joint texture recognition to reduce encoding complexity. First, a classification decision method based on the global and local texture features of the CU is proposed, efficiently dividing the CU into smooth and complex texture regions. Second, for the CUs in smooth texture regions, the partition is determined by terminating early. For the CUs in complex texture regions, a proposed CNN is used for predictive partitioning, thus avoiding the traditional recursive approach. Finally, combined with texture classification, the proposed CNN achieves a good balance between the coding complexity and the coding performance. The experimental results demonstrate that the proposed algorithm reduces computational complexity by 61.23%, while only increasing BD-BR by 1.86% and decreasing BD-PSNR by just 0.09 dB.

The scope of this research lies in the combination of pre-trained Convolutional Neural Networks (CNNs) and Quantum Convolutional Neural Networks (QCNN) in application to Remote Sensing Image Scene Classification(RSISC). Deep learning (RL) is improving by leaps and bounds pretrained CNNs in Remote Sensing Image (RSI) analysis, and pre-trained CNNs have shown remarkable performance in remote sensing image scene classification (RSISC). Nonetheless, CNNs training require massive, annotated data as samples. When labeled samples are not sufficient, the most common solution is using pre-trained CNNs with a great deal of natural image datasets (e.g., ImageNet). However, these pre-trained CNNs require a large quantity of labelled data for training, which is often not feasible in RSISC, especially when the target RSIs have different imaging mechanisms from RGB natural images. In this paper, we proposed an improved hybrid classical–quantum transfer learning CNNs composed of classical and quantum elements to classify open-source RSI dataset. The classical part of the model is made up of a ResNet network which extracts useful features from RSI datasets. To further refine the network performance, a tensor quantum circuit is subsequently employed by tuning parameters on near-term quantum processors. We tested our models on the open-source RSI dataset. In our comparative study, we have concluded that the hybrid classical–quantum transferring CNN has achieved better performance than other pre-trained CNNs based RSISC methods with small training samples. Moreover, it has been proven that the proposed algorithm improves the classification accuracy while greatly decreasing the amount of model parameters and the sum of training data.

The risks involved in nighttime driving include drowsy drivers and dangerous vehicles. Prominent among the more dangerous vehicles around at night are the larger vehicles which are usually moving faster at night on a highway. In addition, the risk level of driving around larger vehicles rises significantly when the driver’s attention becomes distracted, even for a short period of time. For the purpose of alerting the driver and elevating his or her safety, in this paper we propose two components for any modern vision-based Advanced Drivers Assistance System (ADAS). These two components work separately for the single purpose of alerting the driver in dangerous situations. The purpose of the first component is to ascertain that the driver would be in a sufficiently wakeful state to receive and process warnings; this is the driver drowsiness detection component. The driver drowsiness detection component uses infrared images of the driver to analyze his eyes’ movements using a MSR plus a simple heuristic. This component issues alerts to the driver when the driver’s eyes show distraction and are closed for a longer than usual duration. Experimental results show that this component can detect closed eyes with an accuracy of 94.26% on average, which is comparable to previous results using more sophisticated methods. The purpose of the second component is to alert the driver when the driver’s vehicle is moving around larger vehicles at dusk or night time. The large vehicle detection component accepts images from a regular video driving recorder as input. A bi-level system of classifiers, which included a novel MSR-enhanced KAZE-base Bag-of-Features classifier, is proposed to avoid false negatives. In both components, we propose an improved version of the Multi-Scale Retinex (MSR) algorithm to augment the contrast of the input. Several experiments were performed to test the effects of the MSR and each classifier, and the results are presented in experimental results section of this paper.

This paper presents a novel method based on a convolutional neural network to recover thermal time constants from a temperature–time curve after thermal excitation. The thermal time constants are then used to detect the pathological states of the skin. The thermal system is modeled as a Foster Network consisting of R-C thermal elements. Each component is represented by a time constant and an amplitude that can be retrieved using the deep learning system. The presented method was verified on artificially generated training data and then tested on real, measured thermographic signals from a patient suffering from psoriasis. The results show proper estimation both in time constants and in temperature evaluation over time. The error of the recovered time constants is below 1% for noiseless input data, and it does not exceed 5% for noisy signals.

Optimizing the performance of heating, ventilation, and air-conditioning (HVAC) systems is critical in today’s energy-conscious world. Fan coil units (FCUs) play a critical role in providing comfort in various environments as an important component of HVAC systems. However, FCUs often experience failures that affect their efficiency and increase their energy consumption. In this context, deep learning (DL)-based fault detection offers a promising solution. By detecting faults early and preventing system failures, the efficiency of FCUs can be improved. This paper explores DL models as fault detectors for FCUs to enable smarter and more energy-efficient hotel buildings. We tested three contemporary DL modeling approaches: convolutional neural network (CNN), long short-term memory network (LSTM), and a combination of CNN and gated recurrent unit (GRU). The random forest model (RF) was additionally developed as a baseline benchmark. The fault detectors were tested on a real-world dataset obtained from the sensory measurement system installed in a hotel and additionally supplemented with simulated data via a physical model developed in TRNSYS. Three representative FCU faults, namely, a stuck valve, a reduction in airflow, and an FCU outage, were simulated with a much larger dataset than is typically utilized in similar studies. The results showed that the hybrid model, integrating CNN and GRU, performed best for all three observed faults. DL-based fault detectors outperformed the baseline RF model, confirming these solutions as viable components for energy-efficient hotels.

Neurological disorders have an extreme impact on global health, affecting an estimated one billion individuals worldwide. According to the World Health Organization (WHO), these neurological disorders contribute to approximately six million deaths annually, representing a significant burden. Early and accurate identification of brain pathological features in electroencephalogram (EEG) recordings is crucial for the diagnosis and management of these disorders. However, manual evaluation of EEG recordings is not only time-consuming but also requires specialized skills. This problem is exacerbated by the scarcity of trained neurologists in the healthcare sector, especially in low- and middle-income countries. These factors emphasize the necessity for automated diagnostic processes. With the advancement of machine learning algorithms, there is a great interest in automating the process of early diagnoses using EEGs. Therefore, this paper presents a novel deep learning model consisting of two distinct paths, WaveNet–Long Short-Term Memory (LSTM) and LSTM, for the automatic detection of abnormal raw EEG data. Through multiple ablation experiments, we demonstrated the effectiveness and importance of all parts of our proposed model. The performance of our proposed model was evaluated using TUH abnormal EEG Corpus V.2.0.0. (TUAB) and achieved a high classification accuracy of 88.76%, which is higher than in the existing state-of-the-art research studies. Moreover, we demonstrated the generalization of our proposed model by evaluating it on another independent dataset, TUEP, without any hyperparameter tuning or adjustment. The obtained accuracy was 97.45% for the classification between normal and abnormal EEG recordings, confirming the robustness of our proposed model.

High-precision land cover maps of remote sensing images based on an intelligent extraction method are an important research field for many scholars. In recent years, deep learning represented by convolutional neural networks has been introduced into the field of land cover remote sensing mapping. In view of the problem that a convolution operation is good at extracting local features but has limitations in modeling long-distance dependence relationships, a semantic segmentation network, DE-UNet, with a dual encoder is proposed in this paper. The Swin Transformer and convolutional neural network are used to design the hybrid architecture. The Swin Transformer pays attention to multi-scale global features and learns local features through the convolutional neural network. Integrated features take into account both global and local context information. In the experiment, remote sensing images from UAVs were used to test three deep learning models including DE-UNet. DE-UNet achieved the highest classification accuracy, and the average overall accuracy was 0.28% and 4.81% higher than UNet and UNet++, respectively. It shows that the introduction of a Transformer enhances the model fitting ability.

There are some irregular and disordered noise points in large-scale point clouds, and the accuracy of existing large-scale point cloud classification methods still needs further improvement. This paper proposes a network named MFTR-Net, which considers the local point cloud’s eigenvalue calculation. The eigenvalues of 3D point cloud data and the 2D eigenvalues of projected point clouds on different planes are calculated to express the local feature relationship between adjacent point clouds. A regular point cloud feature image is constructed and inputs into the designed convolutional neural network. The network adds TargetDrop to be more robust. The experimental result shows that our methods can learn more high-dimensional feature information, further improving point cloud classification, and our approach can achieve 98.0% accuracy with the Oakland 3D dataset.

Speaker Recognition (SR) is a common task in AI-based sound analysis, involving structurally different methodologies such as Deep Learning or “traditional” Machine Learning (ML). In this paper, we compared and explored the two methodologies on the DEMoS dataset consisting of 8869 audio files of 58 speakers in different emotional states. A custom CNN is compared to several pre-trained nets using image inputs of spectrograms and Cepstral-temporal (MFCC) graphs. AML approach based on acoustic feature extraction, selection and multi-class classification by means of a Naïve Bayes model is also considered. Results show how a custom, less deep CNN trained on grayscale spectrogram images obtain the most accurate results, 90.15% on grayscale spectrograms and 83.17% on colored MFCC. AlexNet provides comparable results, reaching 89.28% on spectrograms and 83.43% on MFCC.The Naïve Bayes classifier provides a 87.09% accuracy and a 0.985 average AUC while being faster to train and more interpretable. Feature selection shows how F0, MFCC and voicing-related features are the most characterizing for this SR task. The high amount of training samples and the emotional content of the DEMoS dataset better reflect a real case scenario for speaker recognition, and account for the generalization power of the models.

The tremendous growth in online activity and the Internet of Things (IoT) led to an increase in cyberattacks. Malware infiltrated at least one device in almost every household. Various malware detection methods that use shallow or deep IoT techniques were discovered in recent years. Deep learning models with a visualization method are the most commonly and popularly used strategy in most works. This method has the benefit of automatically extracting features, requiring less technical expertise, and using fewer resources during data processing. Training deep learning models that generalize effectively without overfitting is not feasible or appropriate with large datasets and complex architectures. In this paper, a novel ensemble model, Stacked Ensemble—autoencoder, GRU, and MLP or SE-AGM, composed of three light-weight neural network models—autoencoder, GRU, and MLP—that is trained on the 25 essential and encoded extracted features of the benchmark MalImg dataset for classification was proposed. The GRU model was tested for its suitability in malware detection due to its lesser usage in this domain. The proposed model used a concise set of malware features for training and classifying the malware classes, which reduced the time and resource consumption in comparison to other existing models. The novelty lies in the stacked ensemble method where the output of one intermediate model works as input for the next model, thereby refining the features as compared to the general notion of an ensemble approach. Inspiration was drawn from earlier image-based malware detection works and transfer learning ideas. To extract features from the MalImg dataset, a CNN-based transfer learning model that was trained from scratch on domain data was used. Data augmentation was an important step in the image processing stage to investigate its effect on classifying grayscale malware images in the MalImg dataset. SE-AGM outperformed existing approaches on the benchmark MalImg dataset with an average accuracy of 99.43%, demonstrating that our method was on par with or even surpassed them.