Search

As one of the most critical variables in the hot rolling process, the accuracy of rolling force prediction is directly associated with production stability and product quality. Purely data-driven approaches, however, are severely constrained by the quantity and quality of data, posing challenges for further enhancing the accuracy of rolling force prediction. In this paper, a theory fusion deep neural network (DNN) modelling approach was proposed and applied to the prediction of rolling force during hot plate rolling. In terms of model establishment, the novel NN structure was designed in consideration of the rolling mechanism, and senior variable inputs were added at shallow locations in the network to reduce the loss of critical information. In terms of model training, the method of using rolling theory to guide the initialization of the model was proposed to enable the model to learn the theoretical features more completely in the pre-training phase. Finally, a method to optimize the overall structure of the model using the sparrow search algorithm (SSA) was proposed to ensure the best prediction performance. The model was tested with the data in the developed platform, and the results indicated that the proposed method achieves the best accuracy and stability in this paper, and the response relationship between model inputs and output was consistent with existing theoretical knowledge. Thus, the model can be trusted and flexibly applied to the actual manufacturing processes. [ABSTRACT FROM AUTHOR]

Transformers are Deep Neural Networks (DNN) that utilize a self-attention mechanism to capture contextual relationships within sequential data. Unlike traditional neural networks and variants of Recurrent Neural Networks (RNNs), such as Long Short-Term Memory (LSTM), Transformer models excel at managing long dependencies among input sequence elements and facilitate parallel processing. Consequently, Transformer-based models have garnered significant attention from researchers in the field of artificial intelligence. This is due to their tremendous potential and impressive accomplishments, which extend beyond Natural Language Processing (NLP) tasks to encompass various domains, including Computer Vision (CV), audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published, spotlighting the Transformer's contributions in specific fields, architectural disparities, or performance assessments, there remains a notable absence of a comprehensive survey paper that encompasses its major applications across diverse domains. Therefore, this paper addresses this gap by conducting an extensive survey of proposed Transformer models spanning from 2017 to 2022. Our survey encompasses the identification of the top five application domains for Transformer-based models, namely: NLP, CV, multi-modality, audio and speech processing, and signal processing. We analyze the influence of highly impactful Transformer-based models within these domains and subsequently categorize them according to their respective tasks, employing a novel taxonomy. Our primary objective is to illuminate the existing potential and future prospects of Transformers for researchers who are passionate about this area, thereby contributing to a more comprehensive understanding of this groundbreaking technology. • The paper presents a comprehensive survey on transformers for deep learning tasks. • The paper conducts a thorough analysis on highly effective models in five domains. • The paper classifies the models based on respective tasks using a proposed taxonomy. • The characteristics of the surveyed models are deeply explored and analyzed. • Future directions and challenges for transformer-based models are deciphered. [ABSTRACT FROM AUTHOR]

Breast cancer is one of the serious diseases that threaten the life of many women worldwide. The seriousness of this disease is that it is often discovered in late stages after a period of its occurrence. This causes a wide spread of the disease and difficulty in its treatment. Another important characteristic of this disease is that it can return again after a period of its treatment. Therefore, predicting the occurrence or recurrence of such disease early is the best solution to have a high cure rate. The main objective of this paper is to improve the prediction performance of the breast cancer recurrence. Many methods have been proposed to predict breast cancer recurrence. However, these methods did not achieve the desired results on one of the most famous datasets in the field of breast cancer recurrence's prediction (i.e., Wisconsin Prognosis Breast Cancer (WPBC) dataset). The highest accuracy achieved using the previous methods is 89.89%. Therefore, this paper provides a framework for improving the prediction of breast cancer recurrence. The proposed framework has the ability to overcome many of the challenges in the existing dataset such as: (a) the problem of imbalance between classes using a data over-sampling technique; and (b) the large number of data dimensions using Principal Component Analysis (PCA), and a wrapper dimensionality reduction technique based on Genetic Algorithm (GA). It also uses the neural network algorithm to fuse the results of two individual classifiers (i.e., Random Forest (RF) and Support Vector Machine (SVM)). Our proposed framework evaluation showed a significant improvement in the predication performance. It achieved an accuracy of 98.3%, area under the curve of 99%, and precision, recall, and f1-measure of 98%. [ABSTRACT FROM AUTHOR]

Cardiovascular diseases are currently the major causes of death globally. Among the strategies to prevent cardiovascular issues, the automated classification of heart sound abnormalities is an efficient way to detect early signs of cardiac conditions leading to heart failure or other, even asymptomatic, complications, quite effective for timely interventions. Despite the significant improvements in this field, there are still limitations due to the lack of solutions, available data-sets and poor (mainly binary — normal vs abnormal) classification models and algorithms. This paper presents a Medical Cyber–Physical System (MCPS) for the automatic classification of heart valve diseases onsite, in a timely manner. The proposed MCPS, indeed, can be deployed into personal and mobile devices, addressing the limitations of existing solutions for patients, healthcare practitioners, and researchers, through an efficient and easy accessible tool. It combines different neural network models trained on a new Italian dataset of 132 adult patients covering 9 heart sound categories (1 normal and 8 abnormal), also validated against two main open-access (Physionet/CinC Challenge 2016 and Korean) datasets. The overall MCPS performance (time, processing and energy resource utilization) and the high accuracy of the models (up to 98%) demonstrated the feasibility of the proposed solution, even with few data. The dataset supporting the findings of this paper is available upon request to the authors. • DB of 132 heart sounds of Italian patients in 9 categories, available upon request. • MCPS for automated HVD classification via hierarchical ML model structures. • Compared six ML classifiers (RF, SVM, NB, LR, DNN, LSTM) for HVD and majority voting. • MCPS validated on two open access datasets (Physionet/CinC 2016 and Korean). • Time, processing and energy efficiency of MCPS validated, with usage guidelines. [ABSTRACT FROM AUTHOR]

Marine ecosystems and their fish habitats are becoming increasingly important due to their integral role in providing a valuable food source and conservation outcomes. Due to their remote and difficult to access nature, marine environments and fish habitats are often monitored using underwater cameras to record videos and images for understanding fish life and ecology, as well as for preserve the environment. There are currently many permanent underwater camera systems deployed at different places around the globe. In addition, there exists numerous studies that use temporary cameras to survey fish habitats. These cameras generate a massive volume of digital data, which cannot be efficiently analysed by current manual processing methods, which involve a human observer. Deep Learning (DL) is a cutting-edge Artificial Intelligence (AI) technology that has demonstrated unprecedented performance in analysing visual data. Despite its application to a myriad of domains, its use in underwater fish habitat monitoring remains under explored. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring. In addition, we provide a comprehensive survey of key deep learning techniques for fish habitat monitoring including classification, counting, localisation, and segmentation. Furthermore, we survey publicly available underwater fish datasets, and compare various DL techniques in the underwater fish monitoring domains. We also discuss some challenges and opportunities in the emerging field of deep learning for fish habitat processing. This paper is written to serve as a tutorial for marine scientists who would like to grasp a high-level understanding of DL, develop it for their applications by following our step-by-step tutorial, and see how it is evolving to facilitate their research efforts. At the same time, it is suitable for computer scientists who would like to survey state-of-the-art DL-based methodologies for fish habitat monitoring. • Marine environments and fish habitats are widely surveyed using underwater cameras. • Deep learning (DL) offers an effective solution for processing underwater videos. • We write a tutorial on the key concepts and steps of DL for underwater monitoring. • We provide a survey of key DL-based visual fish monitoring methods/datasets. • We shed light on challenges/opportunities of DL for underwater fish monitoring. [ABSTRACT FROM AUTHOR]

The conventional standardized theoretical models (such as Webster, Alcelik, Indo-HCM) used for the delay estimation revolve around the mathematical hypothesis and assumptions, making them static with limitations in accommodating the dynamic traffic behaviors. Recent advances in artificial intelligence make machine learning techniques suitable for estimating vehicle control delay compared to conventional methods. This paper demonstrates the application of several machine learning models developed by focusing on the fluctuations of traffic observed at an intersection having heterogeneous traffic conditions in Ahmedabad city for delay estimation. Several parameters are extracted from on-field and video surveys. However, not all parameters are relevant for accurately estimating vehicle control delay. Hence, a feature selection process consisting of several feature-scoring techniques from filter, wrapper, and embedded methods is applied to all the parameters. This process gave insights into the most statistically relevant independent parameters, and out of all the parameters, cycle time was found to be insignificant, with a feature score of 0 from almost all techniques. Hence, it was removed, and then the prominent parameters were then used to build a vehicle control delay model using support vector regression (SVR), K-nearest neighbor (KNN), artificial neural network (ANN), random forest regression (RF), and decision tree regression (DT). Error distribution, standard deviation of errors, coefficient of Determination (R2), and Root Mean Squared Error (RMSE) are the parameters used for evaluating the performance of the machine learning models. RF outperformed them all with a standard deviation of errors, R2, and RMSE of 11.065, 0.926, and 11.081 on testing data. But ANN, KNN, and DT also performed satisfactorily. Compared with conventional standardized theoretical models, all the machine learning models except SVR performed better. [ABSTRACT FROM AUTHOR]

Medical image analysis, particularly the segmentation of White Blood Cells (WBCs), holds critical importance in scrutinizing the quantity and morphology of these cells in smear images, a pivotal step in disease detection. Existing methodologies often fall short in differentiating between healthy and diseased WBCs, emphasizing color components over internal organizational nuances and external morphologies. This paper presents a pioneering contribution through the introduction of an encoder-decoder deep neural network with an indeterminacy fusion-based model explicitly designed for WBC extraction from blood images. In contrast to prior studies, our approach addresses the inadequacies by delving into the intricacies of WBC structure, characterizing object indeterminacy within the neutrosophic set domain. The encoder-decoder network is fortified by integrating WBC indeterminacy as a fusion component, enabling the segmentation of WBCs into distinct nucleus and cytoplasm regions. The efficacy of our methodology is evaluated across three datasets with varying resolutions. Quantitative metrics such as segmentation accuracy and intersection over union (IoU) are employed to assess the proposed network's performance, which consistently outperforms three original encoder-decoder networks. Notably, our model achieves commendable precision rates, with the highest mean segmentation accuracy reaching 0.95301 across the three datasets. These results underscore the enhanced performance of our segmentation network, attributing its success to the influential role of indeterminacy fusion in augmenting conventional segmentation methods. In summary, this work not only introduces a novel solution for WBC segmentation but establishes its superiority over existing methods. The interpretive insights gained from the results highlight the potential of indeterminacy fusion to drive advancements in the realm of medical image segmentation, emphasizing originality, significance, and superior performance metrics. [ABSTRACT FROM AUTHOR]

Research on automatic translation of sign language to verbal languages has been progressively explored in recent years to assist speech and hearing-impaired people in communicating with non-signers. In this paper, a tiny machine learning (TinyML) solution is proposed for sign language recognition using a low-cost, wearable, internet-of things (IoT) device. A lightweight deep neural network is deployed on the edge device to interpret isolated signs from the Indian sign language using the time-series data collected from the motion sensors of the device. The scarcity of labeled training data is addressed by employing the deep transfer learning approach. Here, the knowledge gained from the data collected using the motion sensors of a different device is used to initialize the model parameters. The performance of the model is assessed in terms of classification accuracy and prediction time for different sampling rates and transferring schemes. The model achieves an average accuracy of 87.18% when all the parameters are retrained with just 4 observations of each sign recorded from the motion sensors of the proposed IoT device. The recognized sign is transmitted to a cloud platform in real-time. A mobile application, SignTalk , is also developed, which wirelessly receives the predicted signs from the cloud and displays it as text. Additionally, text-to-speech conversion is also provided on SignTalk to vocalize the predicted sign for better communication. [ABSTRACT FROM AUTHOR]

Skin diseases are reported to contribute 1.79% of the global burden of disease. The accurate diagnosis of specific skin diseases is known to be a challenging task due, in part, to variations in skin tone, texture, body hair, etc. Classification of skin lesions using machine learning is a demanding task, due to the varying shapes, sizes, colors, and vague boundaries of some lesions. The use of deep learning for the classification of skin lesion images has been shown to help diagnose the disease at its early stages. Recent studies have demonstrated that these models perform well in skin detection tasks, with high accuracy and efficiency. Our paper proposes an end-to-end framework for skin lesion classification, and our contributions are two-fold. Firstly, two fundamentally different algorithms are proposed for segmenting and extracting features from images during image preprocessing. Secondly, we present a deep convolutional neural network model, S-MobileNet that aims to classify 7 different types of skin lesions. We used the HAM10000 dataset, which consists of 10000 dermatoscopic images from different populations and is publicly available through the International Skin Imaging Collaboration (ISIC) Archive. The image data was preprocessed to make it suitable for modeling. Exploratory data analysis (EDA) was performed to understand various attributes and their relationships within the dataset. A modified version of a Gaussian filtering algorithm and SFTA was applied for image segmentation and feature extraction. The processed dataset was then fed into the S-MobileNet model. This model was designed to be lightweight and was analyzed in three dimensions: using the Relu Activation function, the Mish activation function, and applying compression at intermediary layers. In addition, an alternative approach for compressing layers in the S-MobileNet architecture was applied to ensure a lightweight model that does not compromise on performance. The model was trained using several experiments and assessed using various performance measures, including, loss, accuracy, precision, and the F1-score. Our results demonstrate an improvement in model performance when applying a preprocessing technique. The Mish activation function was shown to outperform Relu. Further, the classification accuracy of the compressed S-MobileNet was shown to outperform S-MobileNet. To conclude, our findings have shown that our proposed deep learning-based S-MobileNet model is the optimal approach for classifying skin lesion images in the HAM10000 dataset. In the future, our approach could be adapted and applied to other datasets, and validated to develop a skin lesion framework that can be utilized in real-time. [ABSTRACT FROM AUTHOR]

Sequential recommendation aims to predict future user interactions by analyzing dynamic patterns within their historical behavior sequences. Deep neural networks have recently become popular for learning representations of these sequences in the time domain. However, representing users' intentions in the time domain faces challenges such as noise in interactions and sparsity of data. Contrastive learning and representation learning in the frequency domain can mitigate these issues from different perspectives. In this paper, to fully integrate time-domain sequence representations, frequency-domain sequence representations and contrastive learning based on them, we propose a model called T ime– F requency C onsistency based contrastive learning for S equential Rec ommendation (TFCSRec). TFCSRec utilizes a time-domain encoder with a fully connected network and a filter network to extract high-order features and catch pure sequential patterns. Then, a learnable frequency-domain encoder with a recurrent neural network is designed to capture sequential characteristics in the frequency-domain space. Finally, TFCSRec combines a recommendation task and two contrastive learning tasks to optimize the two user representation encoders. Its contrastive learning is designed to minimize a contrastive regularization loss and a time–frequency consistency loss, which for the first time is constructed directly on the time-domain sequence representation and the frequency-domain sequence representation. Experiments on five benchmark datasets show that the proposed TFCSRec model outperforms other sequential recommendation models based on deep neural networks. • Design a time-domain encoder with a filter to extract sequential patterns. • Devise a learnable frequency-domain encoder to capture sequential characteristics. • Propose a time–frequency consistency based contrastive loss for the first time. [ABSTRACT FROM AUTHOR]

• The proposed method had fused multi-level information compensation and U-Net. • The module designed for multi-level information compensation and compression. • The method had fused compressed features and correlation features. Recently, Deep Convolutional Neural Networks have demonstrated high-quality reconstruction in image super-resolution procedure. In this paper, we propose improved image super-resolution reconstruction via multi-level information compensation and U-Net network to address the problem that the image super-resolution reconstruction algorithm based on deep neural networks tends to lose feature information in the feature extraction process, resulting in the lack of texture and edge details in the reconstructed image. Firstly, we design the U-net like network for image super-resolution reconstruction, which performs multi-level feature extraction and channel compression for the input features through the down-channel branch. It fuses the compressed features and extracts the correlation features of different channels through the bottom module, and performs multi-level feature extraction and channel recovery for the compressed correlation features through the up-channel branch. The multi-level information compensation model is then designed to compensate for the information lost in the channel compression process and the information that is difficult to recover in the channel recovery process of U-net like network. The experimental results can show that the proposed algorithm achieves a significant improvement in Peak Signal-to-Noise Ratio and Structure Similarity Index and visual effect compared with the state-of-arts algorithms. The average experimental results of PSNR from proposed method had improved by 1.63 dB, 1.53 dB, 0.97 dB and 0.94 dB compared to SRCNN, HAT, DAT and CARN, respectively. [ABSTRACT FROM AUTHOR]

Remote sensing applications, such as detection and recognition of objects, need a large number of high spatial resolution (HR) images. As a feasible technique, image fusion is considered to produce HR images. However, most of remote sensing image fusion methods based on deep neural network (DNN) are mainly limited by the extraction of spatial and spectral features. These methods ignore the redundancy among these features, which causes spatial and spectral distortions in the fused image. In this paper, we propose a novel image fusion method based on a triple disentangled network (TDNet) with dual attention to reduce the redundancy in the extracted features. In the proposed method, it is assumed that the information in panchromatic (PAN) and low spatial resolution multispectral (LRMS) images can be encoded as the spatial, spectral, and common features. Specifically, we construct a triple-stream network to extract these features. To efficiently model the spatial and spectral information in PAN and LRMS images, local–global attention and interdependency attention are designed and integrated into the network. Then, the redundancy among these features is reduced by disentangled learning, in which these features are recombined to reconstruct the PAN and LRMS images. Besides, these features should complement each other. So, we utilize the maximal coding rate reduction to balance the redundancy and complementarity among them. Finally, all features are recombined to synthesize the high spatial resolution multispectral image. The experimental results demonstrate that the proposed TDNet has a superior performance in terms of qualitative and quantitative evaluations. The code link is https://github.com/RSMagneto/TDNet. • A triple disentangled network with dual attention is proposed for image fusion. • MCR loss is designed to disentangle spatial, spectral and common features in images. • The local–global spatial similarity is captured by the proposed SLGA module. • The spectral correlation among the LRMS image is modeled by the proposed SID module. [ABSTRACT FROM AUTHOR]

Fine-tuning and mask-tuning (or prompt tuning) are two approaches to construct deep neural networks for entity and relation extraction. Fine-tuning based models optimize neural networks with task-relevant objective, in which pre-trained language models (PLMs) are mainly used as external resources to support word embedding. In mask-tuning models, neural networks is optimized by the same pre-training objective in a PLM, which directly outputs verbalized entity type representations. It is effective to utilize potential knowledge of PLMs. In this paper, we propose a recollect-tuning approach, which jointly makes full use of the mechanisms of both fine-tuning and mask-tuning. In this approach, the recollect-tuning iteratively masks tokens in a possible entity span. The classification is based on both the masked token representation and the entity span representation. It is the same as the process to make a decision based on incomplete information. In the training process, the deep network is optimized by task-relevant objective, which strengthens the semantic representation of each entity span. It is effective to learn entity noise-invariant features and take full advantage of potential knowledge of PLMs. Our method is evaluated on three public benchmarks (the ACE 2004, ACE 2005 and SciERC datasets) for the entity and relation extraction task. The result shows significant improvement in the two tasks, outperforming the state-of-the-art performance on ACE04, ACE05 and SciERC by +0.4%, +0.6%, and +0.5%, respectively. • A unified framework combines fine-tuning and mask-tuning for NER and RE extraction. • A recollect-tuning approach proposed can learn noise-invariant entity features. • A grouped mask attention is designed to model the inter-span token interaction. [ABSTRACT FROM AUTHOR]

Optical Character Recognition (OCR) is a method to convert a scanned photo of"handwritten character recognition (HCR) or printed character recognition (PCR)" into a form of digital text. HCR is a version of OCR, which is remarkably modeled to identify handwritten text, while PCR aims at printed text identification. The identification of handwritten characters and digits is more complicated as compared to PCR because of the diversities in human writing styles, stokes, thickness, and curves of characters. Similarly, achievements in several computer vision tasks consider the Convolutional Neural Networks (CNN) to give an end-to-end solution for HCR with huge success. However, the process of significant feature learning for the identification of images is complicated with little data. Hence, this paper aims to develop a new handwritten character and digit recognition model with the incorporation of a deep learning strategy. Initially, the data related to Indian languages are collected from the standard benchmark datasets. Then, the collected data are given into the feature extraction phase 1, where the ResNet 151 is used for extracting the feature set 1. Similarly, the data gathered are considered in the feature extraction phase 2, where the Optimal Ensemble Pattern extraction approach is developed with Local Binary Pattern (LBP), Local Gradient Patterns (LGP), Local Tetra Pattern (LTrP), and Local Vector Pattern (LVP) for extracting the significant patterns from the language data. These extracted patterns are given into the ResNet 151 for getting the feature set 2. Here, the features from ResNet 151 get optimized with the enhanced optimization algorithm with Fitness-based Sail Fish Optimizer (F-SFO). The obtained feature set 1 and optimal feature set 2 are concatenated for performing final recognition. At last, the HCR is done with the help of developed Bi-LSTM-DNN to achieve the enhanced and accurate recognition of handwritten characters of the Indian languages. The performances of character recognition are further improved with the parameter optimization in Bi-LSTM-DNN with the same enhanced F-SFO. Overall result analysis, the accuracy of the designed method attains 95.12%. [ABSTRACT FROM AUTHOR]

• A scheme to recognize the distraction level was proposed. • MLP-AutoEncoder and DPC algorithms were used to define distraction-level labels. • Factors affecting driver distraction level were explored with mixed-model analyses. • A distraction-level prediction model with stacking ensemble learning was made. Driver distraction-level recognition while performing secondary tasks in full-touch in-vehicle information systems (FTIVISs) is essential for the harmonious co-driving of human and intelligent vehicle systems. However, there has been little research on this topic. To respond to this issue, this paper proposes a distraction-level recognition framework with a combination of semi-supervised learning, unsupervised learning, and supervised learning. First, unsupervised learning and semi-supervised is used to divide the collected unlabeled samples of driving distraction behavior into three categories of distraction levels: high, medium, and low. Second, the factors influencing the distraction level are explored through a mixed model analysis. Finally, a stacking-based ensemble learning model is proposed to recognize the driver distraction level by supervised learning, with the influencing factors of the distraction level used as model input parameters. To improve the recognition performance of the stacking model, four heterogeneous classifiers selected as the base classifiers, and principal component analysis (PCA) is introduced in the base classifier layer, which improves the ability of the meta -classifier to process raw features based on overfitting resistance. We conducted a real road experiment under different road and FTIVIS task conditions, and the proposed model performed better than traditional machine learning models. In addition, the model exhibited the greatest advantage when the deep neural network (DNN) algorithm was used as the meta -classifier of the model, with a recognition accuracy of 92.5%. The study findings are significant for developing a human–machine co-driving control strategy and improving vehicle driving safety. [ABSTRACT FROM AUTHOR]

Lithology identification is a crucial task for reservoir characterization and evaluation. There exists an intricate non-linear response between formation lithology and logging data. However, it is difficult to avoid lithology mislabeling due to human error and interpretation coarsening, and label quality can seriously affect the effectiveness of supervised learning. The presence of noisy labels makes it essential to learn with noisy labels. Noise-filtering methods and noise-robust algorithms only concentrate on a singular aspect of data or algorithm. In this paper, hybrid noise label filtering and correction framework for lithology identification (HNFCL) is proposed. Isolation forest is utilized to detect suspicious data, as it is efficient and fast. Baseline classifiers are built by ensemble tree models. In particular, the labels of abnormal data are removed and Tri-training semi-supervised method is introduced to relabel these data, which minimizes the loss of valid training data. Comprehensive experiments of the HNFCL framework, noise filtering methods and deep neural network methods with optimized loss functions were carried out in the industrial application of logging lithology identification. HNFCL achieved average accuracy of 87.94% and 94.93% in two study wells. These results outperformed the noise filtering methods and showed no significant difference from the state-of-the-art method. The correction of noise by HNFCL will provide a prospect for lithology identification applications. [ABSTRACT FROM AUTHOR]

Emergency Department overcrowding is a well-known problem. The consequences are long waiting times for patients, reduced service quality, and the potential for increased mortality rates. Moreover, this problem is escalating with the aging of the population and the chronification of diseases. Being aware of such concerns, Emergency Departments started to conduct patient accountability years ago, and now they are ready to take advantage of the gathered data to improve the quality of service to citizens. However, to achieve this goal, they need the technology that enables the generation of predictive models from such data to improve the planning of their resources. In this paper, we face the task of forecasting patient admissions by using Deep Neural Networks, particularly a modern Attention-based model. As well, historical records do not provide a full picture of how patients attend the emergency service. Relying on the possibilities given by the attention mechanism, we also propose the use of exogenous information such as calendar data, weather, air quality, allergens, and information extracted from the web via Google Trends to enhance admissions prediction accuracy. In this regard, we have tested different configurations of the exogenous variables to isolate which ones provide relevant information that improves the model. In our experiments, we have seen that the calendar, weather, and air quality provide the most valuable information, meanwhile, allergens and Google Trends data appear to be hindering the models' performance. • Using Attention model for forecasting Emergency Department (ED) patient admissions. • Comparing Attention model with LSTM-based networks for ED patients admissions. • Exogenous variables: weather, calendar, air quality, allergens and Google Trends. • Analyzing the impact of each type of exogenous variable in the forecast. • Attention model with calendar data achieves the best results. [ABSTRACT FROM AUTHOR]

The enhanced answer evaluation system is a cutting-edge automated tool that evaluates subjective answers in various contexts, such as educational assessments, surveys, and feedback forms. The proposed system leverages Natural Language Processing (NLP) and deep learning techniques to analyse subjective answers and provide evaluation scores with precision. Students' answers are evaluated based on various criteria, such as keywords, context, relevance, coherence, and similarity. This paper introduces an architecture for a subjective answer evaluator using three main aspects: detection of keywords, similarity matrix, and presence of named entities. It combines the three aspects and provides a final score. It provides a standardized mechanism to score a given user answer compared to the particular model answer without human prejudice. This research aims to transcend traditional methodologies that predominantly utilize keyword or keyphrase scoring (text-based similarity) to determine the final score of an answer without delving into its technical intricacies. The semantic similarity (vector-based) employs vector data representations for score calculation. This approach necessitates partitioning data into multiple vectors for a comprehensive analysis. While text similarity is effective for short answers, its efficacy diminishes as the length of the answer increases. Therefore, this study emphasizes the critical role of similarity scoring and Named Entity Recognition (NER) scoring in evaluating more extended responses based on the stsb-en-main dataset (short answers) and a custom dataset with 190 records. This research reveals its remarkable performance, which excels through a dynamic three-pronged approach: keyword scoring, semantic similarity, and NER scoring with models like Yet Another Keyword Extractor (YAKE), SimCSE and Camembert. These three independent components synergize to produce unmatched results, establishing a new standard in the field. This enhancement led to Root Mean Square Error (RMSE) scores of 0.031 (optimized error rate) and an impressive 71%+ accuracy for our comprehensive system. This achievement surpasses existing works, which typically reached accuracies ranging between 40%–60% for long answers. • Evaluation using a three-pronged approach of Keyword, Semantic & NER scoring. • Semantic scores enhance similarity matrix grading and impact evaluation. • NER scores address proper noun issues. • Deep Neural Network predicts actual marks effectively. • The best-performing LLM models are YAKE, SimCSE and Camembert. [ABSTRACT FROM AUTHOR]

The state-of-the-art performance on entity disambiguation has been reached by deep neural networks. However, the task remains very challenging due to the complexity of natural language. Moreover, the target data distribution is often different from that of training data. In this paper, we address the limitation of deep entity disambiguation from the perspective of misprediction risk. We propose a knowledge-based approach of risk analysis for entity disambiguation, and leverage it to enable adaptive deep learning. The proposed approach generates risk features by extracting evidences from the knowledge base, and then models them as a linearly-weighted random vector where an attention mechanism is used to focus on the most significant components. Finally, it estimates misprediction risk of the aggregated probability distribution via the Conditional Value-at-Risk metric. Furthermore, we demonstrate how to utilize risk analysis results in adaptive deep learning via two-phase training, the first phase fits on labeled training data while the second one minimizes misprediction risk on unlabeled target data. We evaluate the performance of the proposed approach on benchmark datasets through a comparative study. Our thorough experiments demonstrate that it can detect mispredictions more accurately than existing alternatives and can substantially improve the performance of deep learning models. [ABSTRACT FROM AUTHOR]

The burner combustion tuning is a complex problem that has been studied through flame monitoring and characterization. It has been observed that the flame electromagnetic spectrum and flickering contain specific flame information in combustion processes. This information is helpful for combustion stoichiometry tuning on burners. This paper described a method for selecting the best flame feature subset that can be computed from the scanner signal, in order to get the flame index and induce combustion stoichiometry on burners under specific combustion conditions. We propose a method for selecting a reduced subset with only the useful flame features for flame index classification. To extract the most relevant flame features we use a feature subset selection (FSS) algorithm and to determine the combustion state in burners, five flame indices were defined that represent the most common flame states in oil fuel-fired boilers. FSS includes complete, sequential, and random searches in order to eliminate redundant and noisy flame features to decrease the flame feature set dimension. A probabilistic neural network (PNN) algorithm was implemented for flame feature clustering. Signals from the actual flame scanner system and relevant variables from the boiler data acquisition system were used by the algorithms to calculate the burner flame index. A set of parametric tests was done in a heavy oil-fired boiler under well-known flame and index conditions to train and test the flame classifier. The results showed that only the four more relevant features are enough to classify flames with a good performance (92.3% accuracy), which is useful for burner combustion monitoring and optimization. • A new approach to defining/selecting critical features in the combustion process. • A new approach to classifying flames in industrial heavy oil-fired boilers. • The proposed algorithms are tested & evaluated in a real industrial process. • The overall methodology ensures that the combustion process guarantees safety. [ABSTRACT FROM AUTHOR]

Simulation-driven structural health monitoring (SHM) is crucial for enhancing the safety and reliability of structures. Its primary aim is to utilize numerical simulations to design efficient SHM systems, reducing costs while maintaining effectiveness. However, challenges arise due to disparities between simulations and physical measurements, stemming from uncertainties, noise, and simplifications. These challenges also extend to SHM methods using machine learning, particularly deep neural networks (DNNs). This paper bridges the gap between DNNs trained on simulated data and real-world experiments by investigating three transfer learning (TL) techniques: feature augmentation, direct fine-tuning, and boosting-based fine-tuning. The focus is on guided wave damage interaction coefficients (WDICs) as damage-sensitive features, modeled by DNNs for damage identification in thin plates, commonly found in aerospace structures like aircraft fuselages. This classification task compares measured and predicted WDICs in a common guided wave SHM configuration with one actuator and three sensors. Training and extensive testing datasets are generated using advanced finite element simulations and laser Doppler vibrometer experiments. The study demonstrates that TL techniques significantly enhance DNN prediction accuracy and improve damage identification performance for untrained scenarios compared to purely simulation-based DNNs. Among the TL approaches explored, feature augmentation with appropriate activation functions and boosting-based fine-tuning yield the most promising results. Using four out of twelve TL datasets, feature augmentation reduces the simulation-based mean absolute percentage error from 46.80% to 35.98% and increases coefficient of determination R 2 values from 0.02 to 0.72. With six TL datasets and a boosting-based fine tuning, the mean absolute percentage error decreases from 47.42% to 35.20%, maintaining similar R 2 values. While TL substantially enhances prediction accuracy, its impact on damage identification is slightly less pronounced, with improvements in relative classification accuracy by approximately 5% compared to simulation-based DNNs. These findings highlight the efficacy of TL techniques for damage identification in thin plates using WDICs and DNNs, emphasizing the influence of TL training and DNN model variations. Furthermore, they showcase the potential of TL methods to enhance DNN performance in future SHM applications, particularly emphasizing the promise of feature augmentation and boosting. • Novel simulation-driven structural health monitoring technique using guided waves. • Deep neural network modeling to predict wave damage interaction coefficients. • Bridging disparity between simulations and experiments through transfer learning. • Study of transfer learning techniques based on feature augmentation and fine tuning. • Demonstrated significant enhancement in simulation-based neural network performance. [ABSTRACT FROM AUTHOR]

Accurate and automatic segmentation of individual cell instances in microscopy images is a vital step for quantifying the cellular attributes, which can subsequently lead to new discoveries in biomedical research. In recent years, data-driven deep learning techniques have shown promising results in this task. Despite the success of these techniques, many fail to accurately segment cells in microscopy images with high cell density and low signal-to-noise ratio. In this paper, we propose a novel 3D cell segmentation approach DeepSeeded, a cascaded deep learning architecture that estimates seeds for a classical seeded watershed segmentation. The cascaded architecture enhances the cell interior and border information using Euclidean distance transforms and detects the cell seeds by performing voxel-wise classification. The data-driven seed estimation process proposed here allows segmenting touching cell instances from a dense, intensity-inhomogeneous microscopy image volume. We demonstrate the performance of the proposed method in segmenting 3D microscopy images of a particularly dense cell population called bacterial biofilms. Experimental results on synthetic and two real biofilm datasets suggest that the proposed method leads to superior segmentation results when compared to state-of-the-art deep learning methods and a classical method. • Data-driven seed estimation of a classical seeded watershed for 3D dense cell segmentation. • Enhancement of cell interior and border voxels by predicting Euclidean distance transforms using deep learning. • Multi-scale structural similarity index measure (MS-SSIM) loss for image-to- image mapping with better quality output. • Localization of cell seeds by performing voxel-wise classification using a deep network. [ABSTRACT FROM AUTHOR]

The emotional states of speech have a great impact on the efficiency of speaker recognition (SR) system. Many researchers focus on how to map speech with different emotions to an emotion invariant embedding, which reduces the diversity of data. This paper proposes a new emotion embedding framework with self-attention mechanism for speaker recognition. First, several deep neural networks (DNNs) are trained to classify speakers in different emotional states as emotion embedding extractors during development phase. Then at enrollment stage, these pre-trained models are used to extend emotion embeddings from neutral speech. In order to make the final speaker embedding more representative, the classification model is trained with self-attention mechanism in emotion dimension, so that the framework can automatically annotate the weights of the emotion embeddings. Experiments were carried out on both Mandarin Affective Speech Corpus (MASC) and Crowd-Sourced Emotional Multimodal Actors Dataset (CREMA-D). The results show the proposed method achieves the best of Identification Rate (IR) and Equal Error Rate (EER) which are 59.14%, 15.79% on MASC and 75.98%, 8.14% on CREMA-D compared with state-of-the-art methods. In addition, the cross-database experiments also further demonstrate the practicability of the method in real scenes. • An emotion embedding framework for robust emotional speaker recognition is proposed. • Emotional feature extractors is pre-trained to obtain prior emotional representation. • Various emotion embeddings are extracted by decomposing the emotional features. • Self-attention is introduced to measure the importance of emotion embeddings. • Emotional speaker embedding enriches neutral speech with emotional information. [ABSTRACT FROM AUTHOR]

The aim of this paper is to explore the potential of the class of recurrent neural networks in developing a classification model for the stock market. For this research, the data that replicates the entire Nasdaq stock market limit order book is used. After extracting order book attributes from the raw dataset, feature selection approaches based on conditional entropy and Stochastic Universal Sampling are proposed, in order to highlight potentially informative features for our data classification task. Furthermore, the performances of nine presented recurrent neural network models for classifying the stock market data vector into one of the labels from the set S = {buy, sell, idle} are examined. In this study, the performances of the models based on different networks, namely the gated recurrent unit, the long short-term memory, and the recurrent neural network, are compared. Using a multi-criteria approach it is concluded that the model based on the GRU topology fed with the features selected using the newly proposed feature selection method outperforms other examined models. • The performances of nine different limit order book classification models are examined. • The PROMETHEE method specifies the most suitable model among the observed models. • The proposed feature demonstrates superiority compared to other selection methods. • The models based on different neural network topologies are evaluated. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel dimensionality reduction term (i.e. TripletPCA and ContrastivePCA) designed to enhance the robustness of pair-based loss functions in metric learning against image corruptions. Notably, our approach achieves this without the need for any data preprocessing steps. Our method can be seamlessly integrated into existing models, such as a block with Plug & Play framework. To the best of our knowledge, our TripletPCA and ContrastivePCA represent the first attempts to incorporate dimensionality reduction directly into pair-based metric learning losses for end-to-end metric learning. By projecting image features into low-dimensional vectors, our proposed loss functions effectively retain the essential components of images while mitigating the impact of corrupted features. Consequently, our metric learning loss function accurately computes feature distances through the projection. Experimental results demonstrate that our dimensionality reduction term can be easily incorporated into various types of existing deep neural networks. This integration leads to a substantial improvement in performance on standard benchmark datasets for corrupted image classification tasks. On several corruption datasets, we achieve an average performance improvement of 10.55% compared to existing baseline methods. Our code is available at https://github.com/Juryun/TDRL. [ABSTRACT FROM AUTHOR]

Spiking neural networks (SNNs) have garnered significant attention due to their ultra-high-speed and ultra-low-power operation, rendering them suitable for a range of energy-efficient applications. However, their performance in image generation is relatively mediocre, resulting in a lack of an effective generative replay mechanism to handle sequential probabilistic representations of multiple datasets. Consequently, this limitation often leads to catastrophic forgetting. This paper introduces a novel SNN lifelong learning framework namely Dynamic Lifelong learning with Spiking Generative Networks (DL-SGN), aimed at providing better generation ability while mitigating the problem of catastrophic forgetting in a continual learning scenario. DL-SGN comprises three key components: dynamic experts, a student, and an assistant. The dynamic expert is implemented as a dynamically expanding mixture model, with a proposed network expansion mechanism Dynamic Knowledge Adversarial Fusion (DKAF) facilitating the automatic handling of an increasing number of tasks. the student module adopts an SNN-based Variational Autoencoder (VAE) and classifier, leveraging accumulated knowledge from each expert. To enhance the student's ability to generalize to images, we introduce a discriminator as an assistant module trained using adversarial training. We validate the effectiveness of the proposed by conducting experiments on image generation and classification in a lifelong learning environment, and the results substantiate the effectiveness of DL-SGN. • The DL-SGN framework is proposed to addresses catastrophic forgetting in SNNs. • Our model is based on multiple dynamic experts and adversarial training. • Develop a dynamic knowledge adversarial fusion to expand expert module's capacity. • Employ an assistant module to enhance the student module's generalization ability. [ABSTRACT FROM AUTHOR]

Hardware Trojans (HT) are the most malicious components attacking modern integrated circuits (ICs). Information leakage, incorrect functionality, and overheating are the major effects of HT insertion. Due to the outsourcing of multi-stage digital circuit design and fabrication, HTs can be inserted by rogues at every stage. The types of HTs and insertion methods evolve continuously and their detection becomes more complex. Consequently, designing integrated circuits robust to HT insertion seems efficient. A prerequisite for such a strategy is the evaluation of integrated circuit vulnerability to HT insertion. There are several factors involved in the vulnerability of fabricated integrated circuits such as the distribution of white spaces in the IC layout, the ratio of unutilized routing resources, the amount of activity of the circuit's nodes, the geometrical properties of the circuit's gates, the delay of various paths, and etc. The interrelation of these diverse factors makes IC vulnerability modeling a challenging problem. In this paper, a deep neural network (DNN) based approach is developed to combine all the effective factors in IC vulnerability analysis. Firstly, we generate a comprehensive dataset containing various types and sizes of hardware Trojans inserted into benchmark circuits. The benchmark circuits used in this study are ISCAS 85, ISCAS 89, and ITC 99. Then using the transfer learning concept seven state-of-the-art deep convolutional neural networks are trained and verified using the constructed dataset to classify every integrated circuit into three vulnerability levels (low, moderate, and highly vulnerable classes). Simulation results show that our proposed method achieves almost 95 % accuracy in the most successful case (VGG16). The lowest accuracy belongs to Inception V3 (∼87.65 %) which is much better than the most successful previous studies (72 %). This is mainly because the DNN-based method handles shortcomings (such as incomplete modeling of vulnerability-related factors) in the previous state-of-the-art methods efficiently. [ABSTRACT FROM AUTHOR]

3D point clouds are a crucial type of data collected by LiDAR sensors and widely used in transportation applications due to its concise descriptions and accurate localization. Deep neural networks (DNNs) have achieved remarkable success in processing large amount of disordered and sparse 3D point clouds, especially in various computer vision tasks, such as pedestrian detection and vehicle recognition. Among all the learning paradigms, Self-Supervised Learning (SSL), an unsupervised training paradigm that mines effective information from the data itself, is considered as an essential solution to solve the time-consuming and labor-intensive data labeling problems via smart pre-training task design. This paper provides a comprehensive survey of recent advances on SSL for point clouds. We first present an innovative taxonomy, categorizing the existing SSL methods into four broad categories based on the pretexts' characteristics. Under each category, we then further categorize the methods into more fine-grained groups and summarize the strength and limitations of the representative methods. We also compare the performance of the notable SSL methods in literature on multiple downstream tasks on benchmark datasets both quantitatively and qualitatively. Finally, we propose a number of future research directions based on the identified limitations of existing SSL research on point clouds. [ABSTRACT FROM AUTHOR]

As an emerging digital product, artificial intelligence models face the risk of being modified. Malicious tampering will severely damage model functions, which is different from normal modifications. In addition, tampering localization for targeted repair can effectively reduce the cost. Therefore, it is crucial to achieve model content authentication and locate the tampering location. We proposed a novel semi-fragile neural network watermarking method in this paper to address these issues. Specifically, with the precondition of maintaining model performance, we proposed a method that generates a set of semi-fragile samples for a model to achieve the content authentication and tampering localization. The experiment results show that the content authentication of the model can be achieved by analyzing the output results of the model for the semi-fragile samples. When the model is processed normally, the output results are consistent with the expected label, while when the model is maliciously tampered with, the model produces unstable output. Furthermore, the tamper localization of the model can be further achieved through the information hidden in the semi-fragile samples, resulting in an average accuracy of more than 99.42%. In addition, our method is also effective for other deep neural networks. [ABSTRACT FROM AUTHOR]

Phishing websites distribute unsolicited content and are frequently used to commit email and internet fraud. Detecting them before any user information is submitted is critical. Several efforts have been made to detect these phishing websites in recent years. Most existing approaches use hand-crafted lexical and statistical features from a website's textual content to train classification models to detect phishing web pages. However, these phishing detection approaches have limitations, including (1) the tediousness of extracting hand-crafted features, which require specialized domain knowledge to determine which features are useful for a particular platform; and (2) the difficulties encountered by models built on hand-crafted features to capture the semantic patterns in words and characters in URL and HTML content. To address these challenges, this paper proposes WebPhish, an end-to-end deep neural network trained using embedded raw URLs and HTML content to detect website phishing attacks. First, the proposed model automatically employs an embedding technique to extract the corresponding characters into homologous dense vectors. Then, the concatenation layer merges the URL and HTML embedding matrices. Following that, Convolutional layers are used to model its semantic dependencies. Extensive experiments were conducted with real-world phishing data, which yielded an accuracy of 98.1%, showing that WebPhish outperforms baseline detection approaches in identifying phishing pages. • WebPhish employs both raw URL and HTML content to detect phishing web pages. • WebPhish uses character-level embeddings to enable the feature vectors to generalize to new data. • Extensive experiments conducted on a real-world dataset yields 98.1% accuracy. • Its application on the airline Twitter dataset shows the proposed model's flexibility. [ABSTRACT FROM AUTHOR]

Researchers and practitioners have attempted to predict the financial market by analyzing textual (e.g., news articles and social media) and numeric data (e.g., hourly stock prices, and moving averages). Among textual data, while many papers have been published that analyze social media, news content has gained limited attention in predicting the stock market. Acknowledging that news is critical in predicting the stock market, the focus of this systematic review is on papers investigating machine learning and text mining techniques to predict the stock market using news. Using Kitchenham's methodology, we present a systematic review of the literature on intelligent financial market prediction, examining data mining and machine learning approaches and the employed datasets. From five digital libraries, we identified 61 studies from 2015–2022 for synthesis and interpretation. We present notable gaps and barriers to predicting financial markets, then recommend future research scopes. Various input data, including numerical (stock prices and technical indicators) and textual data (news text and sentiment), have been employed for news-based stock market prediction. News data collection can be costly and time-consuming: most studies have used custom crawlers to gather news articles; however, there are financial news databases available that could significantly facilitate news collection. Furthermore, although most datasets have covered fewer than 100K records, deep learning and more sophisticated artificial neural networks can process enormous datasets faster, improving future model performance. There is a growing trend toward using artificial neural networks, particularly recurrent neural networks and deep learning models, from 2018 to 2021. Furthermore, regression and gradient-boosting models have been developed for stock market prediction during the last four years. Although word embedding approaches for feature representation have been employed recently with good accuracy, emerging language models may be a focus for future research. Advanced natural language processing methods like transformers have undeniably contributed to intelligent stock market prediction. However, stock market prediction has not yet taken full advantage of them. • Emerging articles (2015–2021) on news-based stock market prediction are reviewed. • There is a growing trend toward using news headlines for stock market prediction. • Recurrent neural networks are advantageous models for stock market prediction. • Stock market prediction not fully took advantage of Natural language processing yet. • Quarterly and annual reports of companies are authorized credible data sources. [ABSTRACT FROM AUTHOR]

Domain adaptation provides the possibility of utilizing the knowledge gained from an auxiliary domain to accomplish the task in another related domain. In this paper, a common feature representation of these domains is learned through a new proposed idea to reduce inter-domain differences more precisely which results in higher accuracy for unsupervised domain adaptation. To decrease these divergences, the proposed method finds a subspace that reduces data distribution discrepancy between domains. To achieve this goal, minimization of the divergence of the class conditional probability distributions is considered. By considering class conditional distributions, more discriminative information of data is preserved compared to applying marginal distributions. Also, an explicit parametric distribution of the source and target domains is considered to reduce the discrepancy between the two domains data which results in higher accuracy compared to the other relevant domain adaptation methods. Experimental studies on benchmark image classification tasks confirmed our assumptions and indicated the significant improvement given by the proposed method compared to the other state-of-the-art methods. • Aligning the feature distribution of the source and target domains explicitly. • Introducing a new loss function to impose the common distribution on the features. • Reducing the class conditional probability distribution distance between the domains. [ABSTRACT FROM AUTHOR]

Early fault detection, both in equipment and the products in process, is of paramount importance in industrial processes to ensure the quality of the final product, avoid abnormal operating conditions, expensive repairs, and even production process shutdown. The growing complexity of industrial systems and the increase in the amount of available data have encouraged the development of intelligent systems for automatic fault prediction/detection, mainly based on Industry 4.0 technologies and, particularly, those based on deep learning methodologies. However, the vast majority of proposals and research carried out to date define specific solutions for specific cases, which still requires a high level of expert knowledge for scaling the solutions to industrial environments. Actually, one of the major issues towards the industrialization of deep learning solutions is the determination of the optimal, or near-optimal, hyper-parameters. In this paper, we propose a low-level set-up effort intelligent failure detection system that integrates deep neural networks with a Bayesian Optimization algorithm for self-tuning of the system hyper-parameters. In addition, to facilitate the industrialization of the proposal and its incorporation into current industrial systems, we embedded the proposal in our previously formulated and tested Simulai architecture which allows for containing and interaction with multiple and heterogeneous technological components of manufacturing processes. Finally, our proposal is tested in two real cases of a different nature. The obtained results show a successful performance and demonstrate the easy online integration and interaction in a real production system. • Industrialization of artificial intelligence through a self-adjusting system. • Improvement of the decision-making process in the detection of industrial failures. • Optimization of deep learning for the improvement of industrial health and quality. • Demonstration of the proposal's adaptability to real cases of different natures. [ABSTRACT FROM AUTHOR]

This paper attempts to develop an intelligent plate fin-and-tube heat exchanger (PFTHE) design system, which is entirely self-programming, to achieve quickly design. The proposed design system consists of four modules: (1) formulation, (2) optimization, (3) post-processing, and (4) decision-making. The proposed design system is implemented and validated with the application of shape optimization of ellipse tubes of plate-fin heat exchanger. In the formulation module, the physical problem to be studied is mathematized and the main design variables will be determined. In the optimization module, a famous algorithm, non-dominated sorting genetic algorithm of type II (NSGA-II), is embedded in an in-house Multi-concept Heat Transfer (MHT) code to achieve call CFD simulation during optimization process. To reduce computation time, Open multi-processing (OpenMP) is employed. The optimal solutions (Pareto solutions) obtained by the optimization module will be stored in the database and also taken as the input for the post-processing module and the decision-making module. In the post-processing module, Artificial neural network (ANN) is utilized to establish the correlation between design variables and heat transfer performance indicators assisting engineers to quickly design. As a short cut for heat exchanger design, both forward and backward designs have been implemented. Finally, in the decision-making module, technique for order preference by similarity to an ideal solution (TOPSIS) is applied to determine the best compromise solution from Pareto solutions according to the actual requirements provided by users. Results show that the proposed design system could determine a best compromise solution by reducing the pressure drop (80%) of the tube bundle without sacrificing too much heat transfer performance (5%) and also save much time for designers. For the forward design, the ANNs taken six decision variables as inputs are modelled to forecast two objectives has reached acceptable precisions. This research provides a promising tool for PFTHE optimization to improve heat transfer and comprehensive performance, also for quickly design based on historical simulation or experimental results. [ABSTRACT FROM AUTHOR]

This paper presents two novel approaches for uncertainty estimation adapted and extended for the multi-link bus travel time problem. The uncertainty is modeled directly as part of recurrent artificial neural networks, but using two fundamentally different approaches: one based on Deep Quantile Regression and the other on Bayesian neural network. Both approaches use a recurrent neural network to predict multiple time steps into the future, but handle the time-dependent uncertainty estimation differently. We present a novel sampling technique in order to aggregate quantile estimates for link level travel time to yield the multi-link travel time distribution needed for a vehicle to travel from its current position to a specific downstream stop point or transfer site. To motivate the relevance of uncertainty-aware models in the domain, we focus on the connection protection application as a case study: An expert system to determine whether a bus driver should hold and wait for a connecting service, thus ensuring the connection, or break the connection and reduce its own delay. Our results show that the proposed quantile sampling method performs overall best for the 80%, 90% and 95% prediction intervals, both for a 15 min time horizon into the future (t + 1), but also for the 30 and 45 min time horizon (t + 2 and t + 3), with a constant, but very small underestimation of the uncertainty interval (1–4 pp.). However, we also show, that the Bayesian model still can outperform the DQR for specific cases. Lastly, we demonstrate how a simple decision support system can take advantage of our uncertainty-aware travel time models to prioritize the difference in travel time uncertainty for bus holding at strategic points, thus reducing the introduced delay for the connection protection application. • Presents two uncertainty estimation methods for multi-link bus travel time. • Presents a novel quantile sampling technique for multi-link aggregation. • Uses a connection assurance (hold and wait) application as a case study. • Finds that the proposed quantile method performs best for most prediction horizons. [ABSTRACT FROM AUTHOR]

Classification methods that are based on hyperspectral images (HSIs) are playing an increasingly significant role in the processes of target detection, environmental management, and mineral mapping as a result of the fast development of hyperspectral remote sensing technology. Improving classification performance is still a significant problem, however, as a result of the high dimensionality and redundancy of hyperspectral image sets (HSIs), as well as the presence of class imbalance in hyperspectral datasets. In the past few years, convolutional neural networks (CNNs) and graph convolutional networks (GCNs) have achieved good results in HSI classification, but CNNs struggle to achieve good accuracy in low samples, while GCNs have a huge computational cost. To resolve these issues, this paper proposes a Multi-Feature Fusion of 3D-CNN and Graph Attention Network MFFCG. The algorithm consists of two elements: the 3D-CNN, which produces good classification for 3D HSI cube data, and GAT-based encoder and decoder modules that help in improving the classification accuracy of the 3D-CNN. Finally, the multiple features are merged with the help of two neural network models. We further develop two optimized GAT models named GAT1 and GAT2, which are used with different layers of 3D-CNN. Algorithms after merging with 3D-CNN are named MFFCG-1 and MFFCG-2, which produce better classification results then other developed methods. Experiments on three public HSI datasets show that the proposed methods perform better than other state-of-the-art methods using the limited training samples and in low classification time. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have shown advantages in dealing with complex nonlinear problems and have been applied in process monitoring. However, traditional DNN based process monitoring methods face the following problems. Firstly, only utilizing the features of the last layer to characterize industrial processes may lead to information loss. Secondly, due to the existence of irrelevant features, using all the deep features to establish a monitoring model may lead to the degradation of monitoring performance. Therefore, this paper proposes a multi-objective optimization based deep feature multi-subspace partitioning method for process monitoring. Firstly, a DNN model is established based on normal data, and all the deep features that can represent process information are extracted. Next, the residual is combined with the deep features to form a multi-level information representation vector. Then, aiming at the detection accuracy and detection diversity of the subspaces, the optimal multi-subspace partition can be solved with multi-objective optimization algorithm and the validation data (including normal data and fault data). Finally, the detection results of multiple subspaces are integrated through a fusion strategy to realize process monitoring. Applications in two industrial processes demonstrate the effectiveness and advantages of MOO-DFMSP. [ABSTRACT FROM AUTHOR]

Domain adaptation (DA) methods facilitate cross-domain learning by minimizing the marginal or conditional distribution shift between domains. However, the conditional distribution shift is not well addressed by existing DA techniques for the cross-domain regression learning task. In this paper, we propose Multi-Domain Adaptation for Regression under Conditional shift (DARC) method. DARC constructs a shared feature space such that linear regression on top of that space generalizes to all domains. In other words, DARC aligns different domains according to the task-related information encoded in the values of the dependent variable. It is achieved using a novel Pairwise Similarity Preserver (PSP) loss function. PSP incentivizes the differences between the outcomes of any two samples, regardless of their domain(s), to match the distance between these samples in the constructed space. We perform experiments in both two-domain and multi-domain settings. The two-domain setting is helpful, especially when one domain contains few available labeled samples and can benefit from adaptation to a domain with many labeled samples. The multi-domain setting allows several domains, each with limited data, to be adapted collectively; thus, multiple domains compensate for each other's lack of data. The results from all the experiments conducted both on synthetic and real-world datasets confirm the effectiveness of DARC. • We propose a new pairwise similarity preserver loss function for regression tasks. • We propose a new domain adaptation method for regression tasks. • Our proposed method adapts domains with any arbitrary shift, concept, or covariate. • Our proposed method is capable of adapting multiple domains. • The results prove our method's efficiency in solving real-world problems. [ABSTRACT FROM AUTHOR]

The development of blockchain technology has brought prosperity to the cryptocurrency market and has made the blockchain platform a hotbed of crimes. As one of the most rampant crimes, phishing scam has caused a huge economic loss to blockchain platforms and users. In order to address the threat to the financial security of blockchain, this paper proposes a model based on hybrid deep neural network to detect phishing scam accounts, namely LBPS (LSTM-FCN and BP neural network-based Phishing Scam accounts detection model), and verifies its effectiveness on Ethereum. The LBPS model provides a novel approach to analyse transaction records by adopting the BP neural network to obtain the implicit relationship between features extracted from transaction records and the LSTM-FCN neural network to capture the temporal feature from all transaction records of a target account. The experimental results demonstrate that the features selected in this paper could identify phishing scam accounts effectively. Moreover, the LBPS model performs better than the existing methods and baseline models with an F1-score of 97.86%. • A more effective approach to extract features from transaction records. • A model integrates manual feature engineering and transaction records analysing. • Achieves high F1 on the phishing scam accounts detection on Ethereum. • Outperforming the existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the rapid growth of the scale of social media information, it is getting more and more difficult for social media to detect fake news by using manual review. The spread of fake news may misguide the public, cause social panic, and even lead to violence, which could be avoided by using early detection technology to timely identify fake news on social media. Since fake news is often deliberately designed to attract attention, it is difficult for mongers to provide pictures that match the fabricated stories. However, most of existing multi-modal solutions only use the information of images and text, but do not take into account the correlation between them, which limits the effect of model detection effect. In this paper, we proposed a novel Fake News Detection Framework (FNDF) in Sina Weibo based on hybrid feature fusion method. Specifically, a total of 16 features from text, images and users are extracted to distinguish fake news. Moreover, we extract image-text correlation between text and images. Then, a new deep neural network model called Fake News Net (FNN) is built to implement the detection of fake news, which makes use of a pre-training model named Enhanced Representation through Knowledge Integration (ERNIE), a convolution network named Visual Geometry Group (VGG-19), and a Back Propagation (BP) neural network. We validated it on a publicly available dataset, which shows that the F1-score of the FNN model reaches 95.90%, outperforming the state-of-the-art methods by 3.08%. The ablation experiment also proves that the correlation between images and texts increased the F1-score of the model by 3.15%. And the data balancing experiments show that our model still keeps outstanding detection performance when there is less fake news compared to real news, which is closer to the real-world scenario. The research in this paper provides theoretical methods and research ideas for the detection of fake news on social networks. • A framework FNDF extracts a total of 16 features from text, images and users respectively. • The semantic features of text in images are deeply mined. • Based on SiameseNet, this paper proposes a feature of image-text correlation. • A novel deep neural network called FNN is built for hybrid feature fusion. • The results demonstrate that our method outperforms the state-of-the-art models. [ABSTRACT FROM AUTHOR]

• An automated system for railgun launcher damage detection is proposed. • A new data set of railgun inner bore damage is obtained by the designed device. • Adaptive data augmentation and focal loss are used to balance the uneven data set. • YOLOv5 and SOLOv2 are used for the detection and shape extraction of damage. • Results on the data set show the effectiveness of the proposed methods. The inner bore damage affects the launch performance and service life of electromagnetic railgun launcher. Detection and observation of railgun inner bore damage contribute to the study on mechanism and development rules of railgun damage. This paper analyzes five types of typical railgun inner bore damage. Based on the detection requirement for the damages, this paper proposes an automated damage detection system for the inner bore of electromagnetic railgun launcher consisting of data acquisition device and detection algorithms. The proposed device can step inside the inner bore of the railgun launcher to take photos of the inner bore surface automatically. We use the images obtained by the proposed device to build a data set for the training and verification of the detection algorithms. The object detection algorithm You Only Look Once v5 (YOLOv5) is utilized to achieve the rapid detection of railgun inner bore damages. We introduce the adaptive data augmentation and the focal loss to balance out the uneven category distribution of our data set. The result proves that our YOLOv5 model reaches the state-of-the-art level in the railgun inner bore damage detection task, with its mean Average Precision (mAP) of 0.659 and detection speed of 47.6 frames per second (fps). We choose Segmenting Objects by Locations v2 (SOLOv2) to extract the shape of the damage, with the Average Precision of 0.631. We further achieve damage statistics and the model visualization of damage distribution. The experimental results show that the proposed detection system meets the requirements of rapid detection and accurate feature extraction. It provides researchers with an approach for the study of railgun inner bore damage mechanism. [ABSTRACT FROM AUTHOR]

Since most branches of the distribution grid may already be close to their maximum capacity, smart management when charging electric vehicles (EVs) is becoming more and more crucial. In fact, office buildings might not be able to handle several transactions at the same time, especially considering the next generation of fast chargers which are very power expensive. Thus, an efficient charging policy needs to be found. This paper proposes the scheduling of real-time EVs charging through deep reinforcement learning (DRL) techniques. DRL has been chosen because it can adaptively learn from interacting with the surrounding environment. The focus of the optimization is to ensure the completion of the charging transactions in a timely manner, while shifting the load from the times of peak demand. The novelty of the proposed approach lies in its innovative framework: pools of electric vehicles with different characteristics are categorized using a clustering algorithm, a tree-based classifier has been developed to sort new instances of EVs, and a multilayer perceptron artificial deep neural network has been trained to predict the expected duration of each charging session. These features are used as inputs to the DRL agent, and are mapped into actions that adjust the maximum power associated to each charging station. The model has been compared to a traditional charging algorithm and increasingly challenging scenarios have been considered. Results have shown that the developed algorithm fails less than the baseline, with a reduction of the load due to EVs charging of 80% during peak times. • This paper proposes real-time EVs charging through deep reinforcement learning. • Innovative framework: clustering EVs characteristics, deep NN to predict charging. • Results show reduction of 80% of the load during peak times. [ABSTRACT FROM AUTHOR]

As an attractive research in biometric authentication, Text Independent Speaker Verification (TI-SV) problem aims to specify whether two given unconstrained utterances come from the same speaker or not. As state-of-the-art solutions, end-to-end approaches using deep neural networks seek to learn a highly discriminative speaker embedding space. In this paper, we propose a novel end-to-end approach for speaker embedding learning by focusing on two crucial factors: speaker embedder architecture and objective function. The proposed module in the speaker embedder is composed of an E fficient M ulti-resolution feature R epresentation (EMR) block followed by a M ulti-scale C hannel A ttention F usion (MCAF) block. The EMR effectively addresses the issue of fixed resolution convolutional kernels which commonly used in most embedder architectures. Moreover, the MCAF significantly improves the simple summation-based feature fusion used in residual embedder networks. Regarding the objective function, we conduct the speaker embedding space towards learning the embedding-to-embedding relations, in addition to only embedding-to-training class relations employed by most previous methods. So, we propose to employ a dynamic graph attention network, on top of the proposed embedder to learn all informative relations between embeddings, and then learn both embedder and graph-based networks in an end-to-end manner. We conduct various experiments on a large-scale benchmark dataset called VoxCeleb1&2. The effectiveness of all proposed components is verified through an ablation study. We show the superior or competitive performances of the proposed approach compared to seven well-known embedding architectures and 32 SV systems, regarding two evaluation metrics, EER and minDCF, as well as the number of embedder parameters. [ABSTRACT FROM AUTHOR]

Today's business environment has created a high level of uncertainty and disturbed procedures in supply chains. Suppliers have been often identified as the main source of risks in creating the massive levels of disruptions in supply chains. That is why resilient supplier selection can greatly reduce purchase costs and time delays and can create stability in business practices, thereby increasing competitiveness and customer satisfaction. Pharmaceutical companies play an important key role in the health of society, and these companies are frequently exposed to this disorder. Hence, this paper tries to propose a new integrated approach based on traditional (delivery, quality, price, technology level) and resilient criteria for supplier selection in pharmaceutical companies using the Z-number data envelopment analysis (Z-DEA) model and artificial neural network (ANN). In the proposed approach, expert opinions have been provided based on Z-numbers due to the inherent ambiguity and uncertainty in the evaluation process. This is the first study that evaluates the pharmaceutical industry based on traditional and resilience factors by presenting a methodological structure under the uncertainty environment. Here, a fuzzy mathematical model is used. A real case study is utilized to indicate the applicability of the proposed approach to resilient supplier selection in the pharmaceutical industry. Finally, the suppliers are ranked and the best supplier is selected regarding the reliable level of α. To indicate the features and capabilities of the selected approach, the performance analysis is presented in three parts. First, the obtained results are compared with a fuzzy DEA (FDEA) method in the form of validation and verification. Second, a sensitivity analysis is executed to show the effects of different criteria on ranking results, and the price index is identified as the most important evaluation criteria. Third, a predictive model is presented based on ANN that is able to detect the efficiency or inefficiency of suppliers with an 83% accuracy. [ABSTRACT FROM AUTHOR]

Deep Neural Networks (DNNs) have powerful learning abilities on high-rank and non-linear features, and thus have been applied to various fields, exhibiting higher discrimination performance than conventional methods. However, their applications in enterprise credit rating tasks are rare, as most DNNs employ the "end-to-end" learning paradigm, producing high-rank representations of objects or predictive results without any explanations. This "black box" approach makes it difficult for users in the financial industry to understand how these predictive results are generated, or what correlations exist with the raw inputs, leading to a lack of trust to the predictions. To address this issue, this paper proposes a novel network to explicitly model the enterprise credit rating problem using DNNs and attention mechanisms, allowing for explainable enterprise credit ratings. Experiments conducted on real-world enterprise datasets show that the proposed approach achieves higher performance than conventional methods, while also providing insights into individual rating results and the reliability of model training. The code is provided on. [ABSTRACT FROM AUTHOR]

Illegal practices that cause stock prices to vary from their fair values are known as stock price manipulations. Our prior study used unsupervised deep learning to detect these unlawful activities, in which the autoencoder model could catch five out of six cases in the Stock Exchange of Thailand (SET). Trading actions of good governance equities in a limit order book format were used to train the model, and abnormalities were viewed as manipulation. This unsupervised technique has the strength of discovering never-before-seen patterns, although it has a low recall rate. In this work, the findings were improved by incorporating knowledge about the popular pump-and-dump pattern into another supervised deep learning model, which has a strength in catching known patterns. To combine the advantages of both models, an ensemble of supervised and unsupervised deep neural networks was proposed. The main contribution of this paper is to demonstrate that the new ensemble model achieved a higher detection rate than the unsupervised model alone. The supervised learning model used was a long short-term memory (LSTM) network, the unsupervised learning model used was an LSTM-based autoencoder, and the stack classifier in the ensemble model was a support vector machine (SVM). The proposed method was evaluated on six real manipulation cases from the SET. The empirical results showed that our ensemble model could improve the detection strength, achieved high accuracy on pump-and-dump unseen data, and could identify six out of six real manipulation cases with low false alarm rates. [ABSTRACT FROM AUTHOR]

• Ensemble framework is proposed to state estimate and control under uncertainty. • Dynamically linearizing ensemble learning models help to autonomous control. • Ensemble different source of models is benefit to accuracy of state control. Reactor autonomous control is one of the key research fields of advanced reactor R&D. Although some basic achievements have been made, it is still an unfinished goal to realize the intelligent autonomous control of the reactor. In this paper, some learning-based and ensemble-based predictive models and some model predictive control strategies combined with ensemble Kalman filter (EnKF) are proposed to realize a feasible path to the reactor autonomous control. For a typical point reactor and the operation target, reactor state space predictions are completed by mechanism-based models and data-driven learning-based models respectively. Mechanism models based on parameter perturbation and the neural network models that have higher accuracy are selected as predictors of model predictive control (MPC). The MPC results show that the model with less precision (e.g. support vector machine and random forest regression) has large oscillation results, while the predictive control based on neural network and mechanism model can effectively achieve the control objectives. To improve the accuracy of MPC, ensemble learning methodologies with a second learner based on linear regression, stacking different predictive models into one model are proposed. The accuracy of MPC with the ensemble model can exceed or be close to the optimal individual results. EnKF is proposed to estimate core state and covariance, which would be helpful to restrain the influence of observation uncertainties and predictive model error. In addition, some limitation of the proposed methodologies is discussed and needed further study. [ABSTRACT FROM AUTHOR]

In recent years different types of Residual Neural Networks (ResNets, for short) have been introduced to improve the performance of deep Convolutional Neural Networks. To cope with the possible redundancy of the layer structure of ResNets and to use them on devices with limited computational capabilities, several tools for exploring and compressing such networks have been proposed. In this paper, we provide a contribution in this setting. In particular, we propose an approach for the representation and compression of a ResNet based on the use of a multilayer network. This is a structure sufficiently powerful to represent and manipulate a ResNet, as well as other families of deep neural networks. Our compression approach uses a multilayer network to represent a ResNet and to identify the possible redundant convolutional layers belonging to it. Once such layers are identified, it prunes them and some related ones obtaining a new compressed ResNet. Experimental results demonstrate the suitability and effectiveness of the proposed approach. • Representation of a ResNet model through a multilayer network. • An approach to compress a ResNet through its multilayer network representation. • A compression approach tested on ResNet20,ResNet56,ResNet110 and CIFAR10, CIFAR100. [ABSTRACT FROM AUTHOR]

Collaborative training of deep neural networks using edge devices has attracted substantial research interest recently. The two main architecture approaches for the training process are centrally orchestrated Federated Learning and fully decentralized peer-to-peer learning. In decentralized systems, edge devices, known as agents, collaborate in a peer-to-peer architecture, avoiding the need for a central system to orchestrate the process. Decentralized peer-to-peer (P2P) learning techniques are well researched under the assumption of independent and identically distributed (IID) data across the agents. IID data is seldom observed in real-world distributed systems, and the training performance varies significantly with non-IID data. This paper proposes a decentralized learning variant of the P2P gossip averaging method with Batch Normalization (BN) adaptation for P2P architectures. It is well-known that BN layers accelerate the convergence of the non-distributed deep learning models. Recent research confirms that Federated Learning methods benefit from using the BN method with some aggregation alterations. Our work demonstrated BN effectiveness in P2P architectures by mitigating the non-IID data characteristics across decentralized agents. We also introduce a variant of the early stopping technique that, combined with BN layers, acts as a fine-tuning technique for agent models. We validated our approach by conducting numerous simulations of different model-topology-communication combinations and comparing them to other decentralized baseline approaches. The evaluations were conducted on the next word prediction task using user comments from the Reddit and StackOverflow datasets representing comments from two different domains. Simulations showed that our approach, on average, achieves a mean relative top accuracy increase of 16.9% in ring (19.9% for Reddit, 13.9% for StackOverflow) and 29.8% in sparse (32.9% for Reddit, 26.6% for StackOverflow) communication topologies compared to the best baseline approach. Our code is available at https://github.com/fipu-lab/p2p_bn. • Batch Normalization layers improve decentralized learning on non-IID data. • P2P-BN improves decentralized learning by fine-tuning the Batch Normalization layers. • P2P-BN produces stable models even in sparse communication topologies. [ABSTRACT FROM AUTHOR]

• A novel score-based attack is proposed to deceive the DNN models. • Using substitute and target model to evaluate the word importance. • Synonym is used for generating adversarial samples. • Adversarial training can help to enhance the robustness of DNN models. Deep neural networks (DNNs) are vulnerable to adversarial attacks, in which a small perturbation to samples can cause misclassification. However, how to select important words for textual attack models is a big challenge. Therefore, in this paper, an innovative score-based attack model is proposed to solve the important words selection problem for textual attack models. To this end, the generation of semantically adversarial examples in this model is adopted to mislead a text classification model. Then, this model integrates the self-attention mechanism and confidence probabilities for the selection of the important words. Moreover, an alternative model similar to the transfer attack is introduced to reflect the correlation degree of words inside the texts. Finally, adversarial training experimental results demonstrate the superiority of the proposed model. [ABSTRACT FROM AUTHOR]