Search

In microseismic monitoring, various types of vibration events are often collected. Realizing the automatic identification of microseismic events in many suspected events is the basis of monitoring timeliness. However, due to the different sampling methods of microseismic data provided by different products, the data often contains different waveform sizes and sampling frequencies. This makes it difficult for existing approaches to be widely used in different projects without data preprocessing. In this paper, we propose the Universal Automatic Classification Network (UACNet), a deep learning approach that automatically identifies microseismic data in engineering without preprocessing. The UACNet model includes multiple convolution layers, adaptive average pooling layers, fully connected layers, and UAC blocks. UAC block is a residual structure with multiple convolutional layers and reset and update gates. The adaptive average pooling layer unifies the input size, and the UAC block functions as a feature extraction network to mine sufficient features from data. We test the proposed UACNet on engineering data and compare it with existing common and advanced methods. As a result, UACNet passed the ablation study, and the classification accuracy of UACNet is 95.62%, which is higher than 89.14% of CNN, 91.24% of ResNet, 91.04% of CapsNet, and 86.16% of RTFN, respectively. Moreover, the influence of waveform size, sampling rate, signal-to-noise ratios, and amplitude on the accuracy of UACNet is analyzed. The results show that UACNet can overcome the influence of these factors and truly realize automatic real-time classification of microseismic signals without preprocessing. [ABSTRACT FROM AUTHOR]

The success of deep learning (DL) based-mechanical fault diagnosis hinges on the high quality of training data. However, it is difficult to acquire high-quality mechanical monitoring data due to data contamination: 1) Monitoring device irregularities, such as sensor malfunction and signal transmission disruption, bring anomalies into the training data; 2) human labour-based data annotation inevitably produces incorrectly labeled data. These two types of data contamination degrade the performance of DL models. To address the aforementioned issue, this paper proposes a graph neural network-based data-cleaning method. In the first stage, a group anomaly detector is designed to identify the presence of anomalous data. This detector incorporates affinity graphs for depicting data groups and subsequently calculates the group anomaly score to determine the abnormal group. In the second stage, a graph clustering model is developed to relabel the mislabeled data. This model takes advantage of the graph neural network's proficiency in handling affinity graphs to prepare clean labels for subsequent network training. Experimental results, conducted on a pump and an industrial robot joint reducer, show the proposed method's ability to effectively detect anomalous data and rectify incorrect labeling, surpassing the performance of baseline methods in mechanical fault diagnosis. [ABSTRACT FROM AUTHOR]

Zero-watermarking is a promising technology for image copyright protection due to its excellent imperceptibility and robustness. Compared with deep learning (DL)-based zero-watermarking schemes which are time-consuming and unable to respond in real-time, the moment-based zero-watermarking schemes do not have the aforementioned drawbacks and have drawn great attention with their strong description ability and geometric invariance. However, most existing moment-based zero-watermarking methods bear a number of drawbacks, in particular: (i) Direct computation of image moments is inefficient and inaccurate; (ii) They have difficulty obtaining the favorable trade-off between robustness and discriminability; (iii) Generating pseudo-random numbers based on traditional logistic map results in their lower security performance. To deal with those issues, this paper introduces a new image zero-watermarking method that relies on fast fractional-order polar linear canonical transform (FFrPLCT) features and asymmetric tent map. Firstly , we propose a novel computation strategy called fast polar linear Canonical transform (FPLCT), to save the calculation time of moments. Secondly, we define the fast fractional-order polar linear Canonical transform (FFrPLCT), which is characterized by the generic nature and time-frequency analysis capability. Thirdly, we present the FFrPLCT mixed low-order features by fully exploiting the time-frequency property of FFrPLCT, to improve both the robustness and discriminability of image representation. Finally, we develop a new image zero-watermarking method using FFrPLCT domain mixed low-order features and asymmetric tent map. Experimental results and comparison with well-known existing methods demonstrate the efficacy and superiority of the proposed image zero-watermarking scheme. [ABSTRACT FROM AUTHOR]

Ship trajectory prediction based on Automatic Identification System (AIS) data has attracted increasing interest as it helps prevent collision accidents and eliminate potential navigational conflicts. Therefore, it is necessary and urgent to conduct a systematic analysis of all the prediction methods to help reveal their advantages to ensure safety at sea in different scenarios. It is particularly important and significant within the context of unmanned ships forming a new hybrid maritime traffic together with manned ships in the future. This paper aims to conduct a comparative analysis of the up-to-date ship trajectory prediction algorithms based on machine learning and deep learning methods. To do so, five classical machine learning methods (i.e., Kalman Filter, Gaussian Process Regression, Support Vector Regression, Random Forest, and Back Propagation Network) and eight deep learning methods (i.e., Recurrent Neural Networks, Long Short-Term Memory, Bi-directional Long Short-Term Memory, Gate Recurrent Unit, Bi-directional Gate Recurrent Unit, Sequence to Sequence, Spatio-Temporal Graph Convolutional Network, and Transformer) are thoroughly analysed and compared from the algorithm essence and applications to excavate their features and adaptability for manned and unmanned ships. The findings reveal the characteristics of various prediction methods and provide valuable implications for different stakeholders to guide the best-fit choice of a particular method as the solution under a specific circumstance. It also makes contributions to the extraction of the research difficulties of ship trajectory prediction and the corresponding solutions that are put forward to guide the development of future research. • Multi-perspective systematic analysis of ship trajectory prediction (STP) research. • Comparative analysis of thirteen methods to reveal their advantages and disadvantages. • Applicability of the thirteen extracted methods under difficult STP circumstances. • New solutions to STP for future hybrid traffic of manned and unmanned ships. [ABSTRACT FROM AUTHOR]

In leather images, precise pore segmentation is a necessary medium for accurate species prediction. However, due to foreground color, texture, size, and boundary variability, traditional methods encounter false segmentation. This paper proposes a deep learning-based novel auto-pore segmentation network (ApSnet). It aims to automatically segment the species-definite pores from novel digital microscopic leather image data. The ApSnet learns the pore regions from raw and the corresponding ground-truth images. The images undergo patch-based image augmentation to fasten the learning process. The depth and the concatenations of the existing Unet are reduced to design the simple and shallow ApSnet. The weighted average loss function strengthens the image understanding to segment the accurate pore pixels. The experimental analysis affirms the superior performance of the proposed ApSnet against Unet, Unet++, and I-Unet. The study also validates ApSnet with KDSB18 and KVasir-SEG datasets for medical image segmentation. It also tests the robustness of ApSnet on the leather images with fuzzy textures. Finally, the k-nearest neighbor (KNN) model learns and classifies the features from the ApSnet segmented images with 97.4% accuracy. Thus, the analysis proves that precise pore segmentation is indispensable in interpreting species-distinct pore features. Therefore, the present work develops a human–computer interactive platform for accurate leather species prediction by designing a computationally efficient ApSnet-based segmentation model. [Display omitted] • Novel auto-pore segmentation network (ApSnet) for digital microscopy leather images. • Simple and shallow ApSnet to segment leather images with repetitive pore pattern. • Outperforms Unet, Unet++, and inception-Unet with 92.17% accurate pore segmentation. • Quantitatively validated for open-source KDSB18 and KVasir-SEG datasets. • Resulted in improved and automatic leather species prediction with 97.4% accuracy. [ABSTRACT FROM AUTHOR]

Convolutional neural networks are now seeing widespread use in a variety of fields, including image classification, facial and object recognition, medical imaging analysis, and many more. In addition, there are applications such as physics-informed simulators in which accurate forecasts in real time with a minimal lag are required. The present neural network designs include millions of parameters, which makes it difficult to install such complex models on devices that have limited memory. Compression techniques might be able to resolve these issues by decreasing the size of CNN models that are created by reducing the number of parameters that contribute to the complexity of the models. We propose a compressed tensor format of convolutional layer, a priori, before the training of the neural network. 3-way kernels or 2-way kernels in convolutional layers are replaced by one-way fiters. The overfitting phenomena will be reduced also. The time needed to make predictions or time required for training using the original Convolutional Neural Networks model would be cut significantly if there were fewer parameters to deal with. In this paper we present a method of a priori compressing convolutional neural networks for finite element (FE) predictions of physical data. Afterwards we validate our a priori compressed models on physical data from a FE model solving a 2D wave equation. We show that the proposed convolutional compression technique achieves equivalent performance in the prediction error as classical convolutional layers with fewer trainable parameters(around 20%) and lower memory footprint. [ABSTRACT FROM AUTHOR]

In the process industries, the complex mechanisms, the many variables with complex interactions, the high uncertainty and errors in instrumentation, etc. make it very difficult to build accurate soft sensor models. Domain knowledge plays a very important role in soft sensor modeling. However, current data-driven methods lack domain knowledge of the specific processes. Therefore, a spatio-temporal deep learning soft sensor modeling framework embedded with domain knowledge is proposed in this paper. First, the time-delays between the key variable and the other process variables are analyzed to align the process variables temporally and select secondary variables. A spatio-temporal structure model (STSM) of the process is then constructed by using domain knowledge to decouple the sub-processes. Finally, a deep Long Short-Term Memory (LSTM) model embedded with the STSM is proposed to learn the characteristics of the spatiotemporal nonlinear dynamic features between the sub-process variables and the key variable in industrial processes. The effectiveness of the framework is verified by prediction of the key indices in two mineral processing processes. [ABSTRACT FROM AUTHOR]

Perceiving the future trend of Vessel Traffic Flow (VTF) in advance has great application values in the maritime industry. However, using such big data from the Automatic Identification System (AIS) for accurate VTF prediction remains challenging. Deep training networks can learn valuable features from extensive historical data. This paper proposes a new learning-based prediction network, improved Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) with similarity grouping, including three views. To effectively enable the training network to capture the temporal and periodic (i.e. a spatial attribute) change characteristics of VTF, the CNN and LSTM are employed to compose spatial and temporal views, respectively. Hence, the original one-dimensional data is transformed into a matrix (hour of the day ✕ day) to adapt the input of the proposed methodology. In practical applications, VTF of multiple adjacent target regions need to be predicted simultaneously, and the changes of VTF in different areas may influence each other. To explore their hidden relationships, the similarity grouping view aims to find the target area that exhibits the most similarity with the VTF change trend of the current research area. Furthermore, similar information is combined with the features generated from the other two views to obtain the prediction results. In summary, the new advantage lies in mining the spatiotemporal attributes of data and fusing the similarity information of adjacent regions. Comparative experiments with eleven other methods on realistic VTF datasets show that the proposed method demonstrates superior prediction accuracy and stability performance. • Multi-perspective systematic analysis of ship trajectory prediction research. • Comparative analysis of eleven methods to reveal their advantages and disadvantages. • Applicability of the eleven extracted methods under difficult circumstances. • New solutions to STP for future hybrid traffic of manned and unmanned ships. [ABSTRACT FROM AUTHOR]

Human-Robot Interaction (HRI) constitutes a demanding research field that integrates artificial intelligence, informatics, robotics, engineering, and human-machine interaction. In the present era, there is an increased focus on natural user interfaces, with particular attention to gestural modalities. This leads to the coordination of modern robotic systems, where real-time integration and interpretation of manual gestures play a vital role in facilitating effective manipulation. Hand gesture recognition holds a crucial role in the field of machine vision, perhaps greatly exacerbated with varying lighting conditions and backgrounds. In this paper, first an Enhanced Anchor-free Network (EAF-Net) is proposed to perform hand gesture recognition in real-time. The EAF-Net is deep single-stage Convolutional Neural Network (CNN) based architecture. The Custom Precise Prediction Function (CPPF) is utilized for a continuous recognition of a specific gesture from a video feed. Then, the 6-axis collaborative Robot is manipulated using the predicted gesture. The EAF-Net model utilizes the Enriched Hourglass as a backbone feature extraction network. The proposed EAF-Net model undergoes training and assessment using the MITI HD-II dataset. The performance of the model is also analyzed for the standard benchmark datasets (NUSHP-II and Senz-3D). The evaluation of the EAF-Net model's performance spans a range of IoU values from 0.5 to 0.95. The EAF-Net model led to higher precision values (AP 0.5) as 99.22%, recall values (AR 0.5) as 98.20%, and F1-Score 0.5 as 98.64% on the MITI HD-II dataset. The EAF-Net model consumed a prediction time of 259 ms on NVIDIA Jetson Nano Processor and 14 ms on NVIDIA Titan X GPU. It aided in implementing a portable high-speed 6-axis Robot control framework to pick and place operation in 3D space. The robot would be able to imitate human hand activities such as hand rotation, hand tilting, picking and arranging items, and so on in the near future. Further reduction in prediction time of the model on the portable processors (NVIDIA Jetson Nano, Raspberry-Pi) could enhance the speed of the Human Machine Interaction process. [ABSTRACT FROM AUTHOR]

People increasingly use social media platforms to express themselves by posting visuals and texts. As a result, hate content is on the rise, necessitating practical visual caption analysis. Thus, the relationship between image and caption modalities is crucial in visual caption analysis. Contrarily, most methods combine features from the image and caption modalities using deep learning architectures with millions of parameters already trained without integrating a specialized attention module, resulting in less desirable outcomes. This paper suggests a novel multi-modal architecture for identifying hateful memetic information in response to the above observation. The proposed architecture contains a novel "multi-scale kernel attentive visual" (MSKAV) module that uses an efficient multi-branch structure to extract discriminative visual features. Additionally, MSKAV utilizes an adaptive receptive field using multi-scale kernels. MSKAV also incorporates a multi-directional visual attention module to highlight spatial regions of importance. The proposed model also contains a novel "knowledge distillation-based attentional caption" (KDAC) module. It uses a transformer-based self-attentive block to extract discriminative features from meme captions. Thorough experimentation on multi-modal hate speech benchmarks MultiOff, Hateful Memes, and MMHS150K datasets achieved accuracy scores of 0.6250, 0.8750, and 0.8078, respectively. It also reaches impressive AUC scores of 0.6557, 0.8363, and 0.7665 on the three datasets, respectively, beating SOTA multi-modal hate speech identification models. [ABSTRACT FROM AUTHOR]

Video Summarization is a video compression/compaction technique to create a shorter yet informative version of original video. Video summarization has offered solutions to plenty of media, user and engineering applications. Though sports video summarization has been an active research topic for some time; there still exists a void for multi-modal, dynamic, generic and domain knowledge based approach for Cricket Sport video summarization. This paper presents a multi-modal video summarization approach to summarize Cricket sport videos. This work captures the domain knowledge acquired from multi-modal (audio-visual) cues. A dual neural network architecture pipeline is proposed to dynamically segment and dynamically summarize Cricket videos for generic target audience. The former Neural Network is grounded on Cricket bowling activity (visual feature) for dynamic video segmentation of Cricket videos. The segments are then forwarded to the latter Neural Network for identification of key segments. The key segment detection module relies on Audio analysis of Cricket video stream to identify exciting, content representative and informative segments as per Cricket domain. Experimental analysis on two novel proposed benchmark datasets, i.e. DPCS (Delivery Play Cricket Sport) image dataset and EXINP (Excited Interval Normal Play) Cricket Dataset (audio based) shows promising results. The results indicate that the proposed multi-modal approach generates exciting, content representative, informative, generic and dynamic summary incorporating domain knowledge of the sport. [ABSTRACT FROM AUTHOR]

In the field of histopathology, the microtomy procedure yields thin sections of tissue embedded in paraffin blocks, then to be further processed for diagnostic purposes. Within microtomy, trimming is an initial but critical process in which the excess paraffin covering the tissue of interest is removed by continuous cutting routines, until the tissue is suitably exposed and ready for sectioning. Trimming is currently a time-consuming process that is manually held by technicians. In this paper, we present a method to automatize this process, by analyzing tissue block surface images resulting from each cyclic cutting routine. Two types of Convolutional Neural Networks (CNNs) were fine-tuned: one for binary segmentation, the other for multi-class classification tasks, by exploring and optimizing lightweight architectures to provide fast analytical results on cost-effective edge computing. Two sequential online conditions followed the CNNs to output the current stage of the block surface and rule if the trimming-end cutting routine was reached. We compared the results obtained through our method with the ones obtained by three skilled technicians processing 75 tissue blocks. The proposed method identified the trimming-end cutting routine approximately as accurate as the technicians did, yielding up to 90% of trimmed blocks of optimal quality and evidencing the potential of this tool in future automated trimming instruments. We deployed our method to an Edge TPU hardware accelerator to showcase its capability to provide immediate and objective results at every microtomy station applied with an ad hoc hardware, potentially guaranteeing a throughput 50% higher than manual trimming. [ABSTRACT FROM AUTHOR]

Current vision-based displacement measurement methods have limitations of requiring manual target and parameter adjustment, along with significant user involvement to achieve the desired results. This paper proposes an advanced and comprehensive vision system based on ZED cameras for full-field three-dimensional (3D) vibration displacement measurement of civil structures. Compared to existing binocular systems consisting of multiple monoculars, ZED binoculars offer greater flexibility on instrument setup and data acquisition. A novel keypoint detection and matching algorithm based on deep learning is used to achieve target-free measurement, which is capable of capturing feature points with high precision. The performance of the proposed vision-based approach is evaluated through experimental studies on a six-story scaled frame structure. Statistical analysis of the comparative results in sinusoidal cases demonstrates that the displacement responses obtained from the proposed vision-based approach show consistent results in compliance with displacement sensor measurements. System identification of the target structure is achieved through a seismic case and the error analysis is also concluded in detail. [ABSTRACT FROM AUTHOR]

Lakes have a crucial impact on natural disaster prevention, resource recycling, maintenance of agricultural production and daily life. The traditional way of acquiring lake water body information lacks efficiency, is dangerous, and is not suitable for lake water body information acquisition and real-time monitoring. For this reason, the automated lake water body extraction method based on deep learning semantic segmentation model is gradually becoming a mainstream method. However, most of the semantic segmentation models used for lake extraction today express features through static semantics, while ignoring the extraction relationships of different convolutional kernels for these features. In order to better extract lake water bodies from remote sensing images, this paper proposes a hybrid semantic segmentation method based on K-Net, which achieves high accuracy extraction of lake water bodies by introducing dynamic semantic kernels to iteratively refine the feature information. The superiority of the K-Net-based hybrid model on a Google remote sensing image dataset of lakes is validated. The experimental results show that (1) the hybrid model is able to achieve accurate extraction of lake water bodies, with the UperNet + K-Net model using Swin-l performing the best among all six evaluation metrics, with mean intersection over union (mIoU) reaching 97.77%; and that (2) after incorporating the K-Net module, all tested models obtain a larger mIoU than before. • A K-Net-based semantic segmentation method is proposed for extracting lake water bodies. • The UperNet + K-Net hybrid model achieved the best result. • The results proved the validity and superiority of the proposed model. [ABSTRACT FROM AUTHOR]

This study proposes an accurate asthma detection model using an attention network and machine learning technique. The objective of this study is the automated detection of asthma using an attention network. The lung sounds from 203 subjects involving 767 segments from asthma and 722 segments from healthy subjects were collected using a stethoscope. A novel Attention ResNet18-based deep feature engineering model has been developed in five phases: preprocessing, training the Attention ResNet18 network, extracting deep features, iterative feature selection, and classification using k-nearest neighbor (kNN) or support vector machine (SVM). Gradient-weighted class activation mapping (Grad-CAM) was used to generate heat maps, effectively distinguishing asthma lung sounds from those of normal individuals. By using Grad-CAM, explainable results have been presented. Our proposed model obtained an accuracy of 99.73% using SVM with 10-fold cross-validation, surpassing the performance obtained by previous models. Hence, the developed model has the potential to detect asthma in the real-world scenario. scenario to detect. • A new attention deep learning network was proposed. • We presented a XAI model in this paper. • New respiratory sound dataset was collected to detect Asthma. • Our model attained over 99% classification accuracy. • Our attention-based XAI model outperformed. [ABSTRACT FROM AUTHOR]

With the increasing demand for efficient image dehazing solutions in computer vision algorithms, particularly for autonomous systems, a research focus on low-inference time yet high-performance dehazing techniques has emerged. Existing dehazing methods predominantly rely on either daytime or nighttime haze models, limiting their effectiveness in handling haziness under varying lighting conditions. To address this limitation, this research paper introduces the Light Invariant Dehazing Network (LIDN), an end-to-end image dehazing network consisting of four sub-modules: Feature Extractor, Deep Global Atmospheric Light Estimator, Medium Transmission Extractor, and Encoder-Decoder. The proposed model, trained using Quadruplet loss, effectively reduces artifacts and produces sharper dehazed images. Extensive experiments conducted under diverse lighting conditions demonstrate the superior performance of the proposed LIDN model compared to state-of-the-art daytime and nighttime dehazing approaches. Remarkably, the proposed model achieves these exceptional results with a runtime of only 0.24s per image, making it highly efficient than existing dehazing algorithms. • Light Invariant Dehazing Network (LIDN), an end-to-end image dehazing network. • Operates effectively in various lighting scenarios, both day and nighttime. • LIDN effectively reduces artifacts and produces sharper dehazed images. • Trained using novel Quadruplet loss, reduces artifacts, produces sharper images. • High-performance compared to state-of-the-art technique with a runtime of 0.24s/image. [ABSTRACT FROM AUTHOR]

This paper proposes a spatiotemporal graph neural network capable of effective representation learning of the spatiotemporal interrelationships and interdependencies of in-situ observation data from multiple locations for multivariate multi-step ahead time-series forecasting. The propose model is largely composed of graph learning, spatial encoder, and temporal decoder, and ablation studies on variants of the three modules and comparative experiments with state-of-the-art deep neural networks for sequence modeling were also performed extensively. The proposed model showed improved predictability than conventional numerical model-based approaches or state-of-the-art models by applying consecutive multi-step ahead time-series prediction of sea surface temperature at multiple locations along the coast. For more rigorous performance evaluation, not only the overall performance of the test data, but also the performance of extreme cases included in the test data based on historical records were separately assessed. The prediction rationales were also presented through quantified relative contributions between neighbor locations using the trained adjacency matrix obtained through graph learning. The results showed that it is well consistent with the ocean physics and geographical domain knowledge, demonstrating the feasibility and reliability of the proposed method. Therefore, the proposed method shows sufficient potential to be used as a scientific tool for decision-making in extreme events such as marine heat waves or for operational ocean forecasting. [ABSTRACT FROM AUTHOR]

Deep-learning-based identification and diagnosis of faults in industrial assets has received much attention. It does not need a deep understanding of the target domain or complex signal-processing techniques for time-consuming feature identification and selection. The vast majority of approaches developed in this field rely on public data recorded in laboratory conditions with high-performance measurement equipment. Moreover, primarily vibration data is used for condition monitoring tasks, which is particularly sensitive to typical error patterns in rotating machinery. These conditions are difficult to maintain in industrial environments since using high-performance measurement systems would not be economically feasible. This paper demonstrates that modern deep learning can achieve extraordinary fault detection results with industrial standard low-cost measurement systems. We determine measurements taken from the industrial-sensory equipment that potentially capture the system's fault patterns. Hereafter, we apply traditional failure detection and identification techniques to evaluate the suitability of the taken measurements for bearing fault detection. These techniques apply envelope analysis to reveal the fault patterns in the signals and a support vector machine to separate the fault classes. We can conclude that the used low-cost sensory equipment is capable to capture meaningful, fault-describing patterns. To improve the failure detection performance we are investigating a 2D, as well as 1D, convolutional neural network approach to identify the error patterns and classify the respective errors. We compare the deep-learning-based methods with the traditional methods. Furthermore, we assess which inverter signal carries the largest fault-describing information content. Experimental results indicate that the proposed deep learning methods outperform traditional fault diagnosis methods, hence, demonstrating the effectiveness in an industrial condition monitoring application. [ABSTRACT FROM AUTHOR]

Deep feature search has transpired as a sub-problem of large-scale Reverse Image Retrieval (RIR) algorithms in computer vision. In light of recent progress, transforming deep features into binary codes and employing various search strategies to delineate similar images has gained popularity. However, these binary codes cannot accurately define the similarity between two images, which is essential for image retrieval. Moreover, when retrieving k closest images, the existing RIR approaches show a drop in precision value as k increases. In this paper, we put forward a method, Deep Learning based Improved Reverse Image Retrieval (DLIRIR), which solves the RIR problem by utilizing a deep neural network, clustering methodology, and dimensionality reduction technique. Motivated by the need to prevent information loss in converting deep features to binary codes and reduce search time, we cluster image features and select some features from each cluster called Representative Points (RPs). These RPs are then compared to features of an image (query) to retrieve images similar to it from the database. The Euclidean distances of the query feature and the RPs indicate similarity between the query and the cluster of images represented by that RP. We reduce the number of comparisons by selecting as few RPs as possible. We evaluate the performance of DLIRIR by comparing it with other state-of-the-art algorithms in the RIR domain. Our method achieves a precision greater than 99% in benchmark datasets Caltech 256, Corel 10k, and Adaptiope, while a precision between 80%–85% on large size datasets Cifar 100, Tiny ImageNet, and VGGFace2. • DLIRIR is a reverse image retrieval system (RIR). • DLIRIR is based on deep neural network, clustering and dimensionality reduction. • It improves precision for higher values of k when retrieving top k similar images. • DLIRIR reduces the impact of outliers. • DLIRIR eliminates comparisons with false near images. • Evaluation of DLIRIR is done using six benchmark datasets. [ABSTRACT FROM AUTHOR]

Although research on time series prediction based on deep learning is being actively carried out in various industries, deep learning technology still has a high entry barrier for researchers who have not majored in computer science. This paper presents a tutorial on time series prediction using a deep learning-based model. The entire process of time series data prediction is presented—from data collection to evaluation of prediction results. The details of each step are shown through a case example of predicting peak electricity demand and system marginal price of Jeju Island in Korea using the 1D-CNN and BiLSTM model. In Jeju Island, the proportion of renewable energy in the total power generation capacity is increased to 67% in 2021, requiring more accurate electricity demand forecasts. Therefore, using 808 days of training data from February 2018, electricity demand and SMP for the next 21 days were predicted. To make it easier for readers to follow, the example uses only open public data, and the entire Python source code is shared via a GitHub repository. The prediction error calculated by WRMSSE showed 0.42 in electricity demand and 0.63 in SMP max. A WRMSSE value less than one means that the forecast was relatively good, that is, better than naïve forecasting. This tutorial is not limited to the energy industry but can be utilized for any application requiring time-series data prediction. This article is expected to be of great help to researchers who need to understand the process of time series prediction using deep learning and use it for application in their industry. [ABSTRACT FROM AUTHOR]

Rail corrugation is a common wear mechanism of high-speed railways and subways, which can cause derailment and running noise. However, rail corrugation only has slight texture change on the rail surface, so it is difficult to detect accurately by traditional detection methods. In this paper, a real-time detection method of rail corrugation based on machine vision and a convolutional neural network is proposed, which effectively improves the accuracy and efficiency of rail corrugation detection. Combined with the gray features of each part of the image, a rail surface segmentation method based on the gray maximum value of the sliding window is also proposed. Moreover, the obtained rail surface image is clearer and the feature information of the rail surface can be completely retained, compared with the adaptive threshold segmentation and edge detection segmentation. ShuffleNet V2, a lightweight convolutional neural network, was selected as the corrugation detection model. The squeeze-and-excitation module was integrated into its basic unit to improve its channel attention, and the activation function was re-selected to make the detection have better real-time performance and accuracy. Through experimental verification, the average detection time of a single image of the improved model is 4.01ms, and the detection accuracy is 2.78% higher than that of the unimproved ShuffleNet V2. The research results will be beneficial to the development of the intelligent real-time detection of rail corrugation. [ABSTRACT FROM AUTHOR]

Recently, facial expression recognition (FER) methods have achieved significant progress. However, FER is still challenged by factors such as uneven illumination and low-quality expression images. Exploring the potential of facial expression features to achieve robust FER is particularly important. Inspired by the Transformer's excellent performance in modeling long-range dependencies and in vision tasks, this paper proposes a Transformer Block Enhancement Module (TBEM) for enhancing the feature representation of facial expressions. The proposed module contains a Channel Enhancement (CE) block and a Spatial Enhancement (SE) block. The CE block can adaptively enhance the expression features on the channel dimension by effectively leveraging the channel dependency information, while the SE block enhances the expression features on the spatial dimension by integrating the spatial dependency information. TBEM can output a more robust expression representation by combining CE and SE. The proposed artificial intelligence learning module greatly improves the recognition accuracy of FER engineering tasks. To further illustrate the application of TBEM in real-world FER engineering, three engineering problems are used for verification. Extensive experiments demonstrate that the proposed method improves FER performance by focusing on more accurate decisional features and can be easily embedded into regular convolutional neural network models to help them improve the accuracy on FER tasks by about 2.64%–3.03%. The results show the proposed method achieves accuracies of 90.57% on FERPlus, 89.41% on RAFDB basic and 68.43% on RAFDB compound, respectively. The proposed method also provides a meaningful reference for further research on applying Transformer to other tasks. [ABSTRACT FROM AUTHOR]

Autonomous driving systems require many images for analyzing the surrounding environment. However, there is fewer data protection for private information among these captured images, such as pedestrian faces or vehicle license plates, which has become a significant issue. In this paper, in response to the call for data security laws and regulations and based on the advantages of large Field of View (FoV) of the fisheye camera, we build the first A utopilot D esensitization D ataset, called ADD , and formulate the first deep-learning-based image desensitization framework, to promote the study of image desensitization in autonomous driving scenarios. The compiled dataset consists of 650K images, including different face and vehicle license plate information captured by the surround-view fisheye camera. It covers various autonomous driving scenarios, including diverse facial characteristics and license plate colors. Then, we propose an efficient multitask desensitization network called DesCenterNet as a benchmark on the ADD dataset, which can perform face and vehicle license plate detection and desensitization tasks. Based on ADD , we further provide an evaluation criterion for desensitization performance, and extensive comparison experiments have verified the effectiveness and superiority of our method on image desensitization. [ABSTRACT FROM AUTHOR]

The prediction of the next activity in a business process can be very useful in revealing inefficiencies and take decisions to avoid undesired activities. In this direction, further advantages can gather from the knowledge of the activities within their attributes that influence the occurrence of the predicted activity. This paper faces this issue by introducing a Data-aware Explainable Next Activity Prediction approach called DENAP based on the adoption of Long Short-Term Memory (LSTM) neural networks and Layer-Wise Relevance Propagation (LRP) method for the detection of next activity prediction in a business process and for the evaluation of the past activities and data that influence the prediction. The DENAP approach is validated on a set of synthetic and real logs. The obtained results show the good capability of DENAP to predict the next activity and indicate the more relevant activities/data with respect to the prediction. [ABSTRACT FROM AUTHOR]

Imaging nanoscale radiation-induced defects using the transmission electron microscope (TEM) is a key factor in the successful implementation of materials for nuclear energy structural applications. Analyzing each defect in a TEM micrograph is currently a manual task. To identify the defects in a single image can take anywhere from 15 min to an hour and a project can require the analysis of anywhere from tens to ≥ 100 images. Here, we use artificial intelligence (AI) models to automate this task. For simplification, we evaluated images with only a single type of defect; helium bubbles. We performed semantic segmentation of these helium bubble defects in electron microscopy images of irradiated FeCrAl alloys using a deep learning DefectSegNet model. This model, which was previously used to classify crystal defects, is inspired by the classic DenseNet and U-Net image segmentation models. It claims high spatial resolution, but has poor performance at object boundaries. Our paper improves the DefectSegNet model's application by adding two new features. First, the DefectSegNet model is applied not only to perform calculation pixel-wise but also object (or feature) wise. Because object-wise metrics are directly relevant to our final goal of detecting bubbles, whereas pixel-wise classification is only an intermediate step, it is an important part of our study. Second, a distance map loss (DML) function has been added to increase its performance at object boundaries. It is crucial to accurately represent defects boundaries, especially bubbles, in order to track the bubble-induced swelling caused by irradiation. The boundary-focused DML function is also compared to other loss functions like Cross-entropy, Weighted Binary Cross Entropy (WBCE), Dice and Intersection over Union (IOU). Finally, by incorporating new features, we found a marked improvement on segmentation quality and better shape preservation at the boundaries and areas of the bubbles. [ABSTRACT FROM AUTHOR]

Interior construction makes up a large portion of project budget and time and is more prone to schedule delays. Most research efforts on progress management focus on exterior environment, while few on interior construction. Although progress monitoring methods based on laser point clouds and computer vision are investigated before, the problems of costly acquisition and creation of point clouds and images are still open, which impede the study of progress evaluation, particularly in interior construction environments where clutters and occlusions are universal. This paper introduces a method based on 360° panoramic images and deep learning for fast end-to-end interior progress evaluation in room units. The method takes only one or two 360° panoramic images as input, estimates key corners, generates and registers room layouts, and semantically segments sparse point cloud. With the extracted corners and segmentation results, the progress states of interior trades can be evaluated. The experimental results show that the proposed method based on deep learning techniques achieves comparable performance against those on public data sets with 3D Intersection over Union (3D IoU) of 83.69% vs 84.23%, Corner Error (CE) of 0.4% vs 0.69%, and mean class Interaction over Union (mIoU) of 70.28% vs 53.5%. A case study of an interior decoration project of a hotel is adopted to demonstrate the feasibility and practical capabilities of the proposed method. [ABSTRACT FROM AUTHOR]

The chemical information acquisition capabilities of near infrared (NIR) have attracted great interests of investigators on exploring its potential as process analytical technologies (PAT) on the traditional Chinese medicine (TCM). However, due to the operation environment of TCM is complex and noisy, the accurate online composition detection and analysis is challenging and remains unsolved. Therefore, in this paper, we propose a new online TCM composition analysis platform for high quality online spectral data acquisition. We design, VasLine, a generative framework based on deep learning to estimate the multiple critical quality attributes (CQAs). To demonstrate the feasibility of the system, the platform was applied to the extraction process of Xiao'er Xiaoji Zhike Oral Liquid (XXZOL) and the framework was trained to estimate the content of the 7 CQAs. In the evaluation, we carried out 16 batches extraction experiments and collected extensive online NIR data, offline NIR data and high-performance liquid chromatography (HPLC) data for TCM composition analysis. The results show that the estimation accuracy of VasLine outperforms the state-of-the-art regression approaches significantly for all the 7 different CQAs, e.g., the R2 metrics of VasLine for the CQAs are all higher than 0.95. This study aims to propose a novel generative framework based on a deep learning model, and the framework is applied in the self-developed platform to estimate CQAs with high-quality generative data from noisy online NIR accurately. The experiments for online TCM composition analysis show that VasLine is a new solution for the quality improvement of the actual pharmaceutical industry. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel deep learning approach for predicting saddle points in stochastic two-player zero-sum games. Our method combines neurodynamic optimization and deep neural networks. First, we model the stochastic two-player zero-sum game as an ordinary differential equation (ODE) system using neurodynamic optimization. Second, we develop a neural network to approximate the solution to the ODE system, which includes the saddle point prediction for the game problem. Third, we introduce a specialized algorithm for training the neural network to enhance the accuracy of the saddle point prediction. Our experiments demonstrate that our model outperforms existing approaches, yielding faster convergence and more accurate saddle point predictions. [ABSTRACT FROM AUTHOR]

Driver distraction is one of the main causes of fatal traffic accidents. Therefore, the ability to detect driver inattention is essential in building a safe yet intelligent transportation system. Currently, the available driver distraction detection systems are not widely available or limited to specific class actions. Various research efforts have approached the problem through different techniques, including the usage of intrusive sensors, which are not feasible for mass production. Most of the work in early 2010s used traditional machine learning approaches to perform the detection task. With the emergence of deep learning algorithms, many research has been conducted to perform distraction detection using neural networks. Furthermore, most of the work in the field is conducted under simulation or lab environment, and did not validate the proposed system under naturalistic scenario. Most importantly, the research efforts in the field could be further subdivided into many subtasks. Thus, this paper aims to provide a comprehensive review of approaches used to detect driving distractions through various methods. We review all recent papers from 2014–2021 and categorized them according to the sensors used. Based on the reviewed articles, a simplified framework to visualize the detection flow, starting from the used sensors, collected data, measured data, computed events, inferred behaviour, and finally its inferred distraction type is proposed. Besides providing an in-depth review and concise summary of various published works, the practicality and relevancy of driver distraction detection towards increasing vehicle automation are discussed. Further, several open research challenges and provide suggestions for future research directions are provided. We believe that this review will remain helpful despite the development towards a higher level of vehicle automation. • An overview of usage of machine learning in driver distraction detection is provided. • Deep learning outperforms traditional machine learning in driver distraction detection. • A framework for driver distraction detection is introduced. • Research gap and future research for driver distraction detection is discussed. [ABSTRACT FROM AUTHOR]

Ensemble learning combines several individual models to obtain better generalization performance. Currently, deep learning architectures are showing better performance compared to the shallow or traditional models. Deep ensemble learning models combine the advantages of both the deep learning models as well as the ensemble learning such that the final model has better generalization performance. This paper reviews the state-of-art deep ensemble models and hence serves as an extensive summary for the researchers. The ensemble models are broadly categorized into bagging, boosting, stacking, negative correlation based deep ensemble models, explicit/implicit ensembles, homogeneous/heterogeneous ensemble, decision fusion strategies based deep ensemble models. Applications of deep ensemble models in different domains are also briefly discussed. Finally, we conclude this paper with some potential future research directions. • This paper reviews the state-of-art deep ensemble models and hence serves as an extensive summary for the researchers. • The categorizes of deep ensemble models discussed are bagging, boosting and stacking, negative correlation based deep ensemble models, explicit/implicit ensembles, homogeneous/heterogeneous ensemble, decision fusion strategies, unsupervised, semi supervised, reinforcement learning and online/incremental, multilabel based deep ensemble models. • Application of deep ensemble models in different domains are discussed. • Finally, this paper provides an outlook towards future research directions. [ABSTRACT FROM AUTHOR]

Image colorization, as an essential problem in computer vision (CV), has attracted an increasing amount of researchers attention in recent years, especially deep learning-based image colorization techniques(DLIC). Generally, most recent image colorization methods can be regarded as knowledge-based systems because they are usually trained by big datasets. Unlike the existing reviews, this paper adopts a unique deep learning-based perspective to review the latest progress in image colorization techniques systematically and comprehensively. In this paper, a comprehensive review of recent DLIC approaches from algorithm classification to existing challenges is provided to facilitate researchers' in-depth understanding of DLIC. In particular, we review DLIC algorithms from various perspectives, including color space, network structure, loss function, level of automation, and application fields. Furthermore, other important issues are discussed, such as publicly available benchmark datasets and performance evaluation metrics. Finally, we discuss several open issues of image colorization and outline future research directions. This survey can serve as a reference for researchers in image colorization and related fields. [ABSTRACT FROM AUTHOR]

Accurate multi-node load forecasting is the key to the safe, reliable, and economical operation of the power system. However, the dynamic nature of load and the coupling nature of networks are difficult to extract, making consistent and accurate forecasting of node load rather difficult. In this regard, this paper proposes a soft sharing multi-task deep learning method for multi-node load forecasting in the power system. It has the following aspects: (1) Considering the coupling characteristics of the node network, a multi-modal feature module, based on the inception strategy and gated temporal convolutional network (GTCN), is firstly designed to explore the coupling features implied in the node load data. (2) A novel multi-objective neural network model is proposed to achieve simultaneous prediction of multi-node load by integrating the multi-modal feature module and gated recurrent unit (GRU). For sharing the learning information of sub-networks, this paper uses the soft sharing mechanism to capture load features, which can better optimize the prediction task for each node load simultaneously. Load data from the New Zealand distribution network and AEMO are used to compare the proposed model's performance in various scenarios using regression metrics such as mean absolute percentage error (MAPE), Weighted Mean Accuracy (WMA), root mean squared logarithmic error (RMSLE), and Diebold–Mariano (DM). The simulation results show that the proposed method can explore the spatial–temporal coupling characteristics in multi-node load data. Compared with existing state-of-the-art multi-node load prediction methods, our proposed method's MAPE decrease 17.04% and 3.92% in Non-aggregation and Aggregation situations. • A novel deep multi-task learning method is proposed for the multi-node forecasting problem. • An IGTCN module is designed to extract the coupling features from different node loads. • The multi-task deep neural network exhibits the accuracy of multi-node load forecasting. [ABSTRACT FROM AUTHOR]

In financial engineering, portfolio optimization has been of consistent interest. Portfolio optimization is a process of modulating asset distributions to maximize expected returns and minimize risks. Despite numerous studies on shallow learning models, they have shown limited success in analyzing the complex nature of massive stock data, a task where recent deep learning models excel. However, the deterministic nature of conventional deep learning models impedes their consideration of portfolio risk due to an inherent lack of uncertainty quantification in their predictions. This paper proposes a novel portfolio weighting strategy, incorporating both risk and return considerations within a deep learning framework. We propose the Predictive Auxiliary Classifier Generative Adversarial Networks (PredACGAN), a probabilistic deep learning model, to measure prediction uncertainty. The PredACGAN generator leverages latent vectors and historical stock prices to predict future returns. The model synthesizes predictive distributions from various latent vectors and past prices. The associated risk is produced via the entropy of these distributions, facilitating portfolio optimization through both return and risk considerations. The proposed algorithm removes high-risk assets from the investment universe at rebalancing moments, enabling PredACGAN to optimize portfolios considering both return and risk. We evaluated PredACGAN and the accompanying algorithm with S&P 500 stocks from 1990 to 2020, with portfolios rebalanced monthly based on PredACGAN predictions and risk measures. The PredACGAN portfolios yielded 9.123% annual returns and a 1.054 Sharpe ratio, outperforming a risk-agnostic portfolio yielding 1.024% annual returns and a 0.236 Sharpe ratio. The PredACGAN portfolio also exhibited lower maximum drawdowns, highlighting its effectiveness. • Proposed a novel portfolio optimization algorithm and a deep learning model. • Demonstrated the validity of the proposed algorithm in the stock market context. • Showcased the superior performance of algorithm with S&P 500 and NASDAQ 100 stocks. [ABSTRACT FROM AUTHOR]

Condition monitoring (CM) of wind turbine (WT) has been increasingly adopted for its fault diagnosis and maintenance decision-making. However, the data collected in CM is typically noisy, multidimensional, and highly nonlinear, which causes significant challenges in achieving the effective CM of WT. This paper proposes a novel CM method using a deep learning model with temporal pattern attention (TPA) and a dynamic kernel principal components Mahalanobis distance (DKPMD). The method can evaluate the WT performance accurately for detecting faults. First, outliers are recognized and removed using isolation forest improved by sparse autoencoder and fuzzy c-means clustering (FSIF) from raw wind turbine data of health state for enhancing the quality and reliability of data in modeling. Then, a gated recurrent unit (GRU) is developed for data reconstruction of the objective variables using LassoNet and TPA, which can capture the short- and long-term temporal relationships under different time steps based on selected variables. Meanwhile, kernel RMSE (KRMSE) is applied as a loss function, which avoids the negative effects of large reconstructed errors in parameter optimization. A condition index (CI) is constructed using DKPMD based on the reconstructed errors to consider the dynamic correlation between the variables. Finally, a delay perception-based IF(DPIF) is utilized to determine the threshold. Experiments with data from real WT demonstrate the effectiveness of the developed approach in detecting early abnormal conditions, which outperforms other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Considering the limitation of a single physics-based or deep learning-based method, this paper proposes an effective Dual-model methodology for underwater image enhancement. The Dual-model mainly consists of two branches: the Revised Imaging Network-model (RIN-model) and the Visual Perception Correction-model (VPC-model). Specifically, instead of directly learning an image-to-image mapping strategy, the RIN-model constructs a compact network to estimate the ambient-light and direct-transmission parameters, which are used to reconstruct the preliminary enhanced image based on the revised underwater image formation model. Based on the pre-trained RIN-model, the ambient-light and direct-transmission parameters estimated using only a single input image may be biased, especially for these severely degraded underwater images. To further correct the visual perception of the preliminary enhanced image, the VPC-model incorporates the adaptive-stretching histogram and relative-stretching histogram for contrast enhancement and color restoration, respectively. We also present an adaptive color balance compensation method to estimate the maximum attenuation channel and equalize the information distribution of the three channels in the preliminary enhanced image, which acts as a transition procedure to prevent color distortion or detail destruction caused by excessive stretching in the VPC-model. Comprehensive qualitative and quantitative evaluations have revealed that our Dual-model outperforms seven state-of-the-art underwater image enhancement methods, including the MLLE, Haze-Lines, HLRP, ACDC, UWCNN, FUnIE-GAN, and Water-Net. In the UIEB and EUVP datasets, our methodology achieves relatively better performance than other methods on both full-reference and no-reference evaluation metrics. We also validate that the proposed methodology facilitates the advanced image applications of local feature matching and edge detection. [ABSTRACT FROM AUTHOR]

It is critical for the management and control of urban rail transit (URT) to be able to predict passenger flow accurately and in real time. Considering that the high-resolution data aggregated by the automatic fare collection (AFC) system is wasted, this paper analyzes the problem of applying a multi-time granularity passenger flow data fusion forecasting process. First, we examine the challenge of constructing a dataset of passenger flow data with different time granularities. Thus, an algorithm is proposed for selecting passenger flow datasets with multi-time granularity. Furthermore, a multi-time granularity dense residual network (Mul-DesLSTM) with a dense residual structure and LSTM (long short-term memory) as the predictor is constructed, inspired by a residual network. Using Mul-DesLSTM, finer-grained passenger flow features can be fused layer by layer while maintaining the accuracy of traditional single-granularity passenger flow predictions. Lastly, Mul-DesLSTM is applied to the URT system of Shanghai, China, and compared with baselines. As a result, the proposed Mul-DesLSTM outperforms the baselines with LSTM as a predictor and state-of-the-art model. When the predicted time granularity is 30 min, compared to the single-time granularity LSTM network, the mean absolute error, root mean square error, and symmetric mean absolute percentage error can be reduced by 51%, 63%, and 15%, respectively. The results can serve as a reference and basis for the operation and management of URT systems. [ABSTRACT FROM AUTHOR]

To solve the difficulties in extracting general features of few-shot high-dimensional structural health monitoring data and making accurate repair decision, a civil aircraft structural repair decision method based on deep meta-learning (DML) is proposed in this paper. Firstly, multi-source structural health monitoring data such as component name, cumulative flight hours, inspection level and state parameters are collected as model input. In addition, the damage condition, repair level and repair definition are used as data labels. Secondly, DML model is used for few-shot learning of repair cases. Monitoring data and corresponding labels make up different target datasets, which can be used for training of base-learner and meta-learner. Thirdly, the historical cases in manual are used for in-sample prediction by meta-learning model, while the new cases are utilized for out-of-sample prediction by fine-tuning the initialized deep meta-learning model. Finally, the aircraft aluminum lap joint damage dataset is used for case study. The results show that the proposed deep meta-learning model can achieve higher accuracy when dealing with few-shot high-dimensional input. Compared with other models, the task oriented DML model has more strong generalization ability and higher classification accuracy in analyzing multi-task cases. • Few repair cases with multi-source structural health monitoring data and various repair information are collected to make up few-shot high-dimensional tasks. • Deep meta-learning model based on CNN and MAML is proposed for multitask structural repair decision. • Continuous classification decisions can be achieved by real-time applying and updating of deep meta-learning. [ABSTRACT FROM AUTHOR]

Process mining focuses on the analysis of event log data to build various process analytical capabilities. Predictive process analytics has emerged as one of such key capabilities and it uses machine learning techniques to construct process prediction models. In recent years, deep neural networks have gained increasing interest in process prediction since they can handle multi-dimensional sequential inputs with minimal information loss. However, they are considered black-box models and existing studies in explaining deep neural network-based process predictions rely on only event-level features for explanation. In this paper, we propose a new approach for generating explanations for process outcome predictions at multiple levels. The approach is underpinned by three different prediction models: a transparent model for generating global explanations based on case-level features, an attention-based deep neural network for generating local explanations based on event-level features, and a novel eXplainable Dual-learning Deep network (XD 2 -net) for generating local explanations based on case-level features. Using three publicly available datasets, we have tested the applicability of the approach and further examined the multi-level explanations generated by the approach through an elaborate case study. Unlike others, the design of our approach promotes the idea of leveraging the complementary capabilities of different models and utilizing their strengths, rather than focusing on model performance competition. This will contribute towards generating more comprehensive explanations that meet the needs of different end users and purposes in the future. [ABSTRACT FROM AUTHOR]

Deep support vector data description (DeSVDD) is an emerging anomaly detection method based on the deep learning methodology. However, few studies take the confidence of DeSVDD predictions into account so that the present DeSVDD models cannot indicate the reliability degree of the anomaly detection results. In this paper, we enrich the theory of DeSVDD by building the model confidence definition and developing the corresponding calibration strategy. For one thing, by revisiting the methodology of DeSVDD-based anomaly detection, the confidence of detection results is presented to indicate if the prediction of DeSVDD is reliable. For another, we propose a modified power T -scaling strategy to smooth the anomaly scores of DeSVDD model and improve its calibration performance without changing the original detection results. Six open experiment datasets are used to perform the method testing and the experimental results confirm the effectiveness of our proposed calibration strategy. [ABSTRACT FROM AUTHOR]

With the growing complexity of onboard sensors and the widespread deployment of road sensors, deep learning enables fine-grained traffic prediction using massive amounts of raw traffic data, which facilitates accurate analysis of traffic information in the Internet of Vehicles (IoV). However, most existing studies focus on using all the local data to jointly build a prediction model, facing severe challenges of data security and privacy concerns as well as substantial communication overhead. To address these challenges, in this paper, we propose the Spatial-Temporal Traffic Prediction Network based on federated learning (F-STTP-Net), which only updates the model parameters to the centralized server without any private data. Firstly, we design a sub-area division method, which divides the road network into sub-areas with different macroscopic fundamental diagram properties. Then, we propose a local training model for each sub-area, which uses the graph attention network (GAT) and the long short-term memory (LSTM) to capture the spatio-temporal dependence of the road network. The model uses the branch structure to predict the traffic volume of each intersection in the sub-area. Finally, the local models are aggregated based on federated learning to form a powerful central model, which bridges the constraints on global data sharing and privacy guarantee. We conduct experiments on the real-life dataset in Xuchang Lotus Lake 5G automated vehicles demonstration area to demonstrate that F-STTP-Net can achieve excellent prediction performance without the interaction of sub-area raw data. In addition, the proposed model has a strong generalization ability and can be quickly transferred to a new sub-area. [ABSTRACT FROM AUTHOR]

In this fast-moving digital era, millions of images are added to repositories every millisecond. These images are context-rich images with ample underlying data that are extracted and analyzed by deep neural networks for innumerable research purposes. Significant breakthroughs in deep neural networks are harvesting multi-fold benefits and reaching higher accuracy and prediction rates. Though there is vertical progress in deep learning models there is always a threat to security which disintegrates the strength of deep neural networks. Archived images that an adversary manipulates will still look indistinguishable to the user. Such manipulations called adversarial attacks that trick deep learning models into making wrong predictions are on the rise nowadays. This paper reviews the varied types of adversarial attacks shedding light on their origin, their progressive advancement in multidimensions, along with their applications and challenges. [ABSTRACT FROM AUTHOR]

Anomaly detection in industrial environments aims at detecting anomalies in the monitoring data of industrial machinery or equipment, as soon as possible, preferably presenting real-time alarms, to alert the monitoring staff and start maintenance activities timely. In this paper, the problem of anomaly detection of an industrial furnace is tackled, for the real-time recognition of punctual anomalies on multivariate time series. To this aim, a real-time anomaly detection approach is proposed: first, time series acquired from the real machinery are filtered, to select those of interest for possible anomalies, and pre-processed, to obtain sliding windows for real-time detection, then distinct univariate models are applied, to identify different anomaly types. For the application considered here, data regarding the machinery behaviour were available only for normal functioning, thus an unsupervised approach is chosen. In particular, deep learning models based on autoencoders are used to detect punctual anomalies, by reconstructing each window and evaluating the reconstruction error of its last point. An extensive set of autoencoder models is proposed, with varying architecture in terms of type of model (vanilla/variational autoencoders), type of layers (fully connected/LSTM/BiLSTM), and hyperparameters (number of layers, intermediate sizes, BiLSTM type). Available data are split, and used to train the models, and to test them on the normal signal and on synthetic anomalies injected on it, which are of particular interest and were designed according to domain experts. Performances of the proposed models show differences among them, depending on the model architecture. The most efficient models, in terms of F1 score of detection and number of parameters, are identified by their t-test comparison, and the capability of detecting anomalies online is demonstrated. In particular, the proposed anomaly detection approach, including a selected autoencoder with LSTM layers, is able to correctly recognize normal trends, with very few false positives, and promptly give alarms as different anomalous trends appear. [Display omitted] • Definition and implementation of an online Anomaly Detection approach. • Comparison of different architectures of the autoencoder-based models. • Correct recognition of normal signal and detection of synthetic anomalies. • Application for industrial vacuum furnace predictive maintenance. [ABSTRACT FROM AUTHOR]

This paper proposes an intelligent text classification framework for a resource-constrained language like Bengali, which is considered a challenging task due to the lack of standard corpora, appropriate hyper-parameter tuning method, and pre-trained language-specific embedding. The proposed framework comprises an average meta-embedding feature fusion module and a convolutions neural network module called AVG-M+CNN. This work also proposes an algorithm, i.e., automatic hyperparameter tuning and selection, for enhancing the performance of the AVG-M+CNN technique. All meta-embedding models are evaluated using the intrinsic , e.g., semantic, syntactic, relatedness word similarity, analogy tasks and extrinsic evaluators. The intrinsic evaluator evaluates 200 Bengali semantic, syntactic and relatedness word pairs. Spearman (ρ ˆ), Pearson (r ˆ) and cosine similarity correlations are used to evaluate 18 individual embedding and 9 meta-embedding models. The 3COSADD and 3COSMUL evaluators evaluate the 300 analogy tasks. The extrinsic evaluator evaluates a total of 156 classification models on four corpora: BARD, IndicNLP, Prothom-Alo and B T C C 11 (a newly developed corpus having eleven distinct categories). Among these, the AVG-M+CNN model achieves the highest accuracy regarding four Bengali corpora: 95.92 ±.001% for BARD, 93.10 ±.001% for Prothom-Alo, 90.07 ±.001% for B T C C 11 and 87.44 ±.001% for IndicNLP, respectively. [ABSTRACT FROM AUTHOR]

Pests are diverse and the available datasets often contain an uneven number of examples for different pests (a.k.a., the long-tail distribution). This poses a great challenge to learning-based classification methods, especially deep networks, and often leads to degraded performance, especially for the minority (tail) classes. This paper presents a deep learning integration architecture based on decoupling training and fusion learning, which integrates different models with complementary performance on pest datasets with a long-tailed distribution to improve the overall classification performance of pests. A deep neural network is designed that fuses two complementary deep learning models at the feature level, which consists of a convolution neural network (ConvNeXt) and a Swin Transformer model for decoupling training. Experiments are conducted using three datasets (d0, insect, and IP102), and evaluation on accuracy, recall, and F1-Score is reported. For the large-scale pest dataset with long-tailed distribution IP102, the accuracy achieves 76.1%, which outperforms the state-of-the-art methods. In addition, the accuracy for d0 and insect datasets are 98.5% and 92.3%, respectively. [ABSTRACT FROM AUTHOR]

Single image super-resolution (SISR) aims to recover clear high-resolution images from low-resolution images, which has made great progress with the development of deep learning these years. Scene text image super-resolution (STISR) is a subfield of SISR with the goal of increasing the resolution of a low-resolution text image and enhancing the readability of characters in the image. Despite significant improvements in recent approaches, STISR remains a challenging task due to the diversity of background, text appearances and layouts, etc. This paper presents a Perceiving Multiple Representations (PerMR) method for better super-resolution performances in scene text images. PerMR is a unified network that combines super-resolution with text recognition and exploits the recognizer's feedback to facilitate super-resolution. Specifically, contextual information from the text decoder is extracted to provide sequence-specific guidance and enable the super-resolution model to pay more attention to the text region. Meanwhile, low-level and high-level visual features from the vision backbone of the recognition network are integrated to further improve visual quality. Additionally, we incorporate a frequency branch into the vanilla convolution unit, which efficiently enhances global and local feature representations. Experiments on the STISR benchmark dataset TextZoom validate that PerMR can not only generate more distinguishable images, but also outperforms the current state-of-the-art methods. PerMR boosts the average recognition accuracy by 5.9% using ASTER, 5.8% using MORAN and 10.6% using CRNN compared to the baseline model TSRN. PerMR outperforms the advanced method TPGSR-3 by 1.4% on ASTER, 0.1% on MORAN, 0.2% on CRNN and boosts TATT by 0.6% on ASTER and 1.1% on MORAN respectively. Furthermore, PerMR demonstrates good robustness and generalization when tackling low-quality text images in multiple scene text recognition datasets. The experiment results verify the capabilities of PerMR to boost text recognition performance. [ABSTRACT FROM AUTHOR]

The electrocardiogram (ECG) is a standard test to monitor the activity of the heart. Many cardiac abnormalities are manifested in the ECG including arrhythmia that refers to an abnormal heart rhythm. The basis of arrhythmia diagnosis is the identification of normal versus abnormal heart beats, and their correct classification based on ECG morphology. This paper proposes a novel and robust approach for representation learning of ECG sequences using a LSTM autoencoder for anomaly detection. The encoder part encodes the ECG signal into a lower dimensional latent space representation and decoder part then tries to reconstruct the specified ECG signal. The model is trained only on normal (non-anomalous) ECG signals. Then reconstruction loss of test ECG signals are calculated. Next a reconstruction loss threshold value is determined from the frequency distribution of the reconstruction losses so that from the reconstruction loss value above a certain threshold is determined as anomaly, otherwise it will be treated as normal. Determination of threshold is done using manual and Kapur's automated thresholding procedures. The aforementioned model has been applied on publicly available ECG5000 dataset. From the experimental results it is observed that the proposed model achieved more than 98% accuracy having precision, recall and F1 values more than 0.94, 0.97, 0.96 respectively. The performance of the proposed method is also found to be superior in most of the cases as compared to the results of seven other recent counter-part methods reported in the literature. [ABSTRACT FROM AUTHOR]

Meter error is one of the main contributing factors to unexpected fuel losses or gains in storage tanks at service stations. Although fuel dispensers are expected to be calibrated to standard accuracy periodically to ensure fair and reliable trade in the fuel market, some fuel retailers are unable to keep up with the standards. The current industry practice relies on onsite inspection to identify the issue, which leads to a cost burden if inspections are scheduled too frequently. To the best of our knowledge, there is no previously reported research tailored to the remote meter error detection problem. In this paper, we propose a novel framework for remote and automatic meter error detection via a data-driven approach based on inventory data and fuel transaction data. Specifically, we propose to use mean shift change point detection methods, including statistical-based as well as deep learning-based methods (LSTM-VAE, VAE, Kernel learning), to approach the problem. We present results on our data sets containing both real-world and simulated meter error data, and further evaluate these methods on several widely-used benchmark datasets, to assess their validity, advantages and limitations. The obtained results show that LSTM-VAE outperforms other models in most of the settings for the meter error dataset and the benchmark datasets. [ABSTRACT FROM AUTHOR]

Large-scale model sizes and expensive computing costs cause the challenge of deploying and applying large pre-trained models. Hence, this paper presents a novel Triple Concepts Attention Mechanism and a lightweight TCAMixer model for edge devices to classify texts. Furthermore, the TCAMixer abstracts textual concepts in a human way, which is unmatched by other counterparts such as pNLP-Mixer (a projection-based MLP-Mixer model for Nature Language Processing) and HyperMixer (a hyper network using dynamic token-mixing layers). Experimental results on several public datasets demonstrate that the TCAMixer outperforms the counterparts by a significant margin, for example, achieving 3% higher accuracy with a smaller model size of 0.177M. Additionally, the TCAMixer achieves a performance of 85% to 98.7% compared to that of large pre-trained models but only occupies 1/3000 to 1/2000 of their size on most test datasets. [ABSTRACT FROM AUTHOR]

Grape leaf diseases seriously affect grape yield and directly lead to the loss of the industrial economy. An accurate diagnosis is crucial for effective spraying and control. Presently, the combination of image blending and a deep recognition model is the most effective method for accurately identifying grape leaf diseases. Existing methods of image blending depend primarily on single-channel and multi-scale channel blending, computing and optimizing the Poisson equation, and the iterative optimization of learning-based methods for simple composite results. Although these methods have had some success, they are not able to simultaneously address lesion and leaf gap size, color bleeding, and edge smoothness. In this paper, we propose a novel image blending method named cascade wavelet attention network (CWAN). Specifically, CWAN consists of a cascade wavelet attention (CWA) and a blending autoencoder. First, we build a CWA to decompose the image into a multi-level and multi-scale frequency domain image. Then, the detailed features of the source and target domains are extracted, and the edges are smoothed. Following this, a shared decoder is designed to fuse these features in the corresponding frequency-domain layers, and the result serves as the attention feature. Finally, we design a blending autoencoder with a feature fusion mechanism to seamlessly blend lesions and leaves enhanced by CWA. We also construct a weighted blended loss function to avoid leaf color infiltration into the lesion in the blended image. Comprehensive experiments on two grape leaf disease datasets. Qualitative and quantitative results demonstrate the superiority and effectiveness of our model. • A novel method called CWAN for data enhancement of grape leaf disease is proposed. • The CWA is designed to enhance the textures and smooth the edge of blending. • The blending autoencoder with feature fusion mechanism can seamlessly blend images. • An unpaired image composition dataset is created for grape leaf disease blending. • The image shows superior performance on the texture and seam transition. [ABSTRACT FROM AUTHOR]

In the context of mechanical engineering design, the field of machine learning accelerated topology optimization is dominated by image-based models trained in a supervised manner on datasets with a limited diversity of boundary conditions. State-of-the-art methods show poor generalization capabilities and are strongly coupled to finite element mesh resolution, hindering scalability. In this paper, we leverage the explicit topology parameterization of the moving morphable components (MMC) framework to train a deep learning model that directly generates geometric design variables using a model architecture that is independent of the finite element mesh used for structural analysis. The developed model is trained on a large dataset of boundary conditions. Despite achieving state-of-the-art regression loss, evaluations reveal that direct-design approaches generate topologies with poor mechanical performance. Specifically, the model-generated topologies have, on average, a stiffness 11.48% lower than conventional MMC designs, as evidenced by in-distribution and out-of-distribution test samples. We demonstrate that this is due to the incompatibility between the regression loss function typically used in literature and the topology optimization objective of compliance minimization. To address this issue, we propose a novel acceleration approach that leverages the trained model to generate improved initial designs for conventional optimization. Specifically, the deep learning model is used to generate an initial design, which is then refined by conventional optimization to arrive at a final, optimal design. This approach shows a computation time-saving of 36.84% without sacrificing the final mechanical performance of the optimal topology compared to conventional optimization starting from a uniform initial layout. [ABSTRACT FROM AUTHOR]