Search

• A machine learning approach was used to investigate moisture penetration in paper materials. • The study evaluated the capabilities of both the Feedforward Neural Network (FNN) and the Recurrent Neural Network (RNN). • Numerically generated data was employed for network training. • A continuum model incorporating anisotropic properties, creep behavior, viscoelasticity, and moisture dependency was used for simulations. • The RNN demonstrated superior predictive capability compared to the FNN. In this work, we predicted the gradient of the deformational moisture dynamics in a sized commercial paper by observing the curl deformation in response to the one-sided water application. The deformational moisture is a part of the applied liquid which ends up in the fibers causing swelling and subsequent mechanical response of the entire fiber network structure. The adapted approach combines traditional experimental procedures, advanced machine learning techniques and continuum modeling to provide insights into the complex phenomenon relevant to ink-jet digital printing in which the sized and coated paper is often used, meaning that not all the applied moisture will reach the fibers. Key material properties including elasticity, plastic parameters, viscoelasticity, creep, moisture dependent behavior, along with hygroexpansion coefficients are identified through extensive testing, providing vital data for subsequent simulation using a continuum model. Two machine learning models, a Feedforward Neural Network (FNN) and a Recurrent Neural Network (RNN), are probed in this study. Both models are trained using exclusively numerically generated moisture profile histories, showcasing the value of such data in contexts where experimental data acquisition is challenging. These two models are subsequently utilized to predict moisture profile history based on curl experimental measurements, with the RNN demonstrating superior accuracy due to its ability to account for temporal dependencies. The predicted moisture profiles are used as inputs for the continuum model to simulate the associated curl response comparing it to the experiment representing "never seen" data. The result of comparison shows highly predictive capability of the RNN. This study melds traditional experimental methods and innovative machine learning techniques, providing a robust technique for predicting moisture gradient dynamics that can be used for both optimizing the ink solution and paper structure to achieve desirable printing quality with lowest curl propensities during printing. [ABSTRACT FROM AUTHOR]

With the advancement of science and technology, the number of academic papers published each year has increased almost exponentially. While a large number of research papers highlight the prosperity of science and technology, they also give rise to some problems. As we know, academic papers are the most intuitive embodiment of the research results of scholars, which can reflect the level of researchers. It is also the standard for evaluation and decision-making of them, such as promotion and allocation of funds. Therefore, how to measure the quality of an academic paper is very critical. The most common standard for measuring the quality of academic papers is the number of citation counts of them, as this indicator is widely used in the evaluation of scientific publications. It also serves as the basis for many other indicators (such as the h-index). Therefore, it is very important to be able to accurately predict the citation counts of academic papers. To improve the effective of citation counts prediction, we try to solve the citation counts prediction problem from the perspective of information cascade prediction and take advantage of deep learning techniques. Thus, we propose an end-to-end deep learning framework (DeepCCP), consisting of graph structure representation and recurrent neural network modules. DeepCCP directly uses the citation network formed in the early stage of the paper as the input, and outputs the citation counts of the corresponding paper after a period of time. It only exploits the structure and temporal information of the citation network, and does not require other additional information. According to experiments on two real academic citation datasets, DeepCCP is shown superior to the state-of-the-art methods in terms of the accuracy of citation count prediction. [ABSTRACT FROM AUTHOR]

Highlights are short sentences used to annotate scientific papers. They complement the abstract content by conveying the main result findings. To automate the process of paper annotation, highlights extraction aims at extracting from 3 to 5 paper sentences via supervised learning. Existing approaches rely on ad hoc linguistic features, which depend on the analyzed context, and apply recurrent neural networks, which are not effective in learning long-range text dependencies. This paper leverages the attention mechanism adopted in transformer models to improve the accuracy of sentence relevance estimation. Unlike existing approaches, it relies on the end-to-end training of a deep regression model. To attend patterns relevant to highlights content it also enriches sentence encodings with a section-level contextualization. The experimental results, achieved on three different benchmark datasets, show that the designed architecture is able to achieve significant performance improvements compared to the state-of-the-art. • We propose a novel Transformer-based Highlights Extractor (THExt, in short). • We achieve performance superior to state-of-the-art highlights extraction methods. • Section-level context encoding turns out to be very effective for sentence ranking. [ABSTRACT FROM AUTHOR]

Tremendous academic information causes serious information overload problems while supporting scientific research. Scientific paper and citation recommendation systems have been developed to relieve this problem and work as a filter to furnish only relevant papers to researchers. Although previous studies have made comparative progress, this problem is still challenging because current paper recommendation systems rely on heterogeneous and multi-sourced features, thereby requiring a unified learning representation to cover different types and modalities of information. Additionally, the implicit influence of scholars' previous preferences of writing and citing on his/her new manuscript has not been well considered in the previous studies. Facing the issue from these two aspects, in this paper, a heterogeneous knowledge embedding-based attentive RNN model is proposed to recommend scientific paper citations. First, the preparation of features consists of two parts: (1) building a unified learning representation of structural entities and relations for recommending paper citations; and (2) defining and constructing a bibliographic network comprising five types of entities and five relations. The bibliographic network enables learning a unified representation so that all graphical entities and relations can be vectorized using TransD. To establish textual representations, the PV-DM model is utilized to generate numeric features for the title of each paper. Second, by combining structural and textual representations focusing on the "author-text query" scenario, an attentive bidirectional RNN is constructed to recommend paper and citation based on an user's identity with a length-limited inquiry to capture the scholars' previous writing and citing preferences, thereby reducing recommendation error. Through the DBLP dataset, our experiment results show the feasibility and effectiveness of our method, both in terms of the number as well as the quality of the first few recommended items. In specific, compared with existing models, our model has improved MRR and NDCG by approximately 4.8% and 2.4%, respectively. [ABSTRACT FROM AUTHOR]

This paper suggests a Recalling Enhanced Recurrent Neural Network (RERNN) method for accurately predicting the performance and emissions of spark ignition (SI) engines. The RERNN method is an efficient neural network technique. The RERNN model is also trained using a better conjugate method that speeds up convergence by using a generalized Armijo search technique. The proposed method analyzes combustion phasing, exhaust temperature, engine-out emissions, burn duration, mean effective pressure, ignition lag, and maximum pressure rise rate. By then, the proposed technique is implemented on the MATLAB platform and is evaluated for its performance against existing techniques. The proposed RERNN method achieves a better outcome than other existing Seagull Optimization Algorithm (SOA), Grasshopper Optimization Algorithm (GOA), and Wild Horse Optimizer (WHO) techniques. The proposed method shows the error in carbon monoxide (CO) is 0.18% and carbon dioxide (CO2) is 0.29% at a high speed of 900 rpm compared with other existing approaches. • RERNN method designed for accurately predicting performance in SI engines. • The RERNN method is highlighted as an efficient neural network technique. • The RERNN model is trained using an improved conjugate method. • This approach contributes to a more holistic evaluation of engine performance. • The proposed RERNN method, implemented on the MATLAB platform. [ABSTRACT FROM AUTHOR]

• Review of papers on the diagnosis of brain diseases using the fusion of neuroimaging modalities is presented. • For each brain disease, the most appropriate neuroimaging modalities and best combinations are introduced. • Advantages and disadvantages of fusion levels and different modalities are introduced. • Details of various types of deep neural networks used for the task at hand are also examined. • Finally, future directions and challenges in the field are discussed. Brain diseases, including tumors and mental and neurological disorders, seriously threaten the health and well-being of millions of people worldwide. Structural and functional neuroimaging modalities are commonly used by physicians to aid the diagnosis of brain diseases. In clinical settings, specialist doctors typically fuse the magnetic resonance imaging (MRI) data with other neuroimaging modalities for brain disease detection. As these two approaches offer complementary information, fusing these neuroimaging modalities helps physicians accurately diagnose brain diseases. Typically, fusion is performed between a functional and a structural neuroimaging modality. Because the functional modality can complement the structural modality information, thus improving the performance for the diagnosis of brain diseases by specialists. However, analyzing the fusion of neuroimaging modalities is difficult for specialist doctors. Deep Learning (DL) is a branch of artificial intelligence that has shown superior performances compared to more conventional methods in tasks such as brain disease detection from neuroimaging modalities. This work presents a comprehensive review paper in the field of brain disease detection from the fusion of neuroimaging modalities using DL models like convolutional neural networks (CNNs), recurrent neural networks (RNNs), pretrained, generative adversarial networks (GANs), and Autoencoders (AEs). First, neuroimaging modalities and the need for fusion are discussed. Then, review papers published in the field of neuroimaging multimodalities using AI techniques are explored. Moreover, fusion levels based on DL methods, including input, layer, and decision, with related studies conducted on diagnosing brain diseases, are discussed. Other sections present the most important challenges for diagnosing brain diseases from the fusion of neuroimaging modalities. In the discussion section, the details of previous research on the fusion of neuroimaging modalities based on MRI and DL models are reported. In the following, the most important future directions include Datasets, DA, imbalanced data, DL models, explainable AI, and hardware resources are presented. Finally, the main findings of this study are presented in the conclusion section. [ABSTRACT FROM AUTHOR]

The use of short-term prediction techniques with the purpose of anticipating hyper- or hypoglycemic episodes for patients with type 1 diabetes has gained a lot of interest in the past years. Both hypoglycemia and hyperglycemia have a major impact and considerable effects in the life of a person with diabetes. The prediction of abnormal values of the blood glucose is a vital step in improving the life quality of a diabetic patient. This paper focuses on developing models for predicting blood glucose levels, to warn type 1 diabetes patients about potential hyper- or hypoglycemic events. Our main objective is to help the daily management of people with type 1 diabetes and to improve their long-term health condition. We will compare four models: a regression model, recurrent neural networks, Long Short-Term Memory, and Gated Recurrent Unit. [ABSTRACT FROM AUTHOR]

A multitude of outbreak prediction models are employed on a global scale to facilitate informed decision-making and the implementation of appropriate control measures in light of the COVID-19 pandemic. The authorities are now focusing on a straightforward epidemiological and statistical methodology for forecasting the global COVID-19 epidemic. These models are widely recognized within the media landscape. Long-term prediction has been found to have poor precision in typical models due to the presence of considerable uncertainty and insufficient vital data. Despite several efforts that have been undertaken to tackle this issue, there is a necessity to enhance the fundamental generalization and robustness of current models. There is a need for a comparative analysis of recurrent neural networks (RNN) and long short-term memory models (LSTM) in order to forecast the breakout of COVID-19, in comparison to the currently existing models. The architectural design incorporates the concept of the attention mechanism, specifically, the use of attention inside hidden data. This methodology enables the identification of patterns and the prioritization of time-sensitive information, leading to the creation of a network that is easily comprehensible. This research presents a proposal for utilizing RNN with LSTM to anticipate outbreaks of COVID-19. Additionally, we have incorporated a learning vector time embedding into our methodology, as the inclusion of time in a vector representation may be seamlessly integrated into many architectures. The model provides a forecast of the progression of the COVID-19 pandemic in Europe's most severely affected nations, including Italy, France, Spain, as well as a country in North America, specifically Canada. This study proposes that variations in behavior across different nations indicate the potential efficacy of employing machine learning as a modelling tool for understanding and predicting the spread of epidemics. This paper provides an initial benchmarking analysis to demonstrate the potential of recurrent neural networks (RNNs) for future academic investigations. The study presents a unique method that combines RNNs with LSTM to effectively forecast flare-ups. The experimental results show that our proposed LSTM model performs better than existing models with lesser RMSE and MAPE values. [ABSTRACT FROM AUTHOR]

The concept of predictions has gained much attention over the last few years. Research on prediction based on experience is error-prone. Usually, a lot of data has been available with multiple variables, but not all features are relevant. This work uses different feature selection techniques and their combinations to select the best possible feature subset. This paper uses feature selection techniques for time series data prediction with deep learning models. In this work, we have used filter, wrapper, and embedded methods for feature selection. We combine these techniques and compare their performance with single feature selection techniques. We found that the embedded method alone performs well with fewer features, but its performance deteriorates when the number of features increases. We input these reduced feature sets into deep learning models such as recurrent neural networks (RNN), long short-term memory (LSTM), and 1d Convolutional with LSTM (1d CNN-LSTM). We assess the performance of different deep-learning models with reduced features. Our results analysis found that the right feature subset may help increase the performance of deep learning models for time series prediction. Performance improvement is specific to the dataset, feature selection techniques, and the deployed DL models; performance improvement varies on average, from a few percent to fifty percent. [ABSTRACT FROM AUTHOR]

Traffic congestion has become a critical issue in metropolitan areas, inflicting significant hardships on individuals on a daily basis. This adverse situation poses multifaceted challenges, including increased air pollution, heightened fuel consumption, and compromised road safety. Deep learning algorithms, encompassing both Convolutional Neural Networks (CNN), and Recurrent Neural Networks (RNN) have showcased remarkable performance in areas such as computer vision, natural language processing, healthcare, finance, and autonomous systems. This research paper proposes a novel methodology to develop a vehicle detection model - VehiNet, which can subsequently be refined to create an automated traffic signal system. The study explores the effectiveness of various CNN, RNN, and deep learning algorithms and techniques in accomplishing this objective. The dataset picked for the study in addition to being compiled from internet resources, was also painstakingly assembled manually using a variety of methods to collect over 2000 photographs from various angles, lighting conditions, and locations within Bengaluru city. The main motive is to apply CNN, RNN, and other Deep Learning classification techniques for object detection and classification of Indian vehicles and see how well these deep learning algorithms and techniques affect prediction efficiency. This research stands out from others and makes its novelty clear thanks to the carefully collected dataset and the detailed analysis of the methodologies. [ABSTRACT FROM AUTHOR]

Recurrent neural networks (RNNs) and long short-term memory (LSTM) models have demonstrated tremendous effectiveness in modeling time series data due to the ability they possess to capture temporal dependencies. Stock market data is an example of time series data. Stock price data is very volatile and dynamic therefore one of the most turbulent areas to invest in might be the stock market. The choice to buy or sell stocks is heavily influenced by external circumstances and statistical analysis of previous stock performance. Stock price index forecasting has been a major area of research for many years and many machine learning and deep learning methods have been proposed to simplify this hard process, but little success has been discovered so far. In this paper, the application of RNNs and LSTMs on a stock price dataset to predict future values is explored. We start with Recurrent neural networks (RNNs) to predict the values but one of the major challenges with it is the "vanishing gradient" problem, which makes it difficult for the network to learn long-term dependencies in the data. To overcome this long short-term memory (LSTM) was used which eliminates the vanishing gradient problem. Data were normalized and divided into time steps to determine the relationship between past values and future values to make accurate predictions. The outcomes show that the MLS LSTM method significantly outperforms existing algorithms, achieving a prediction accuracy of 98.1% on the training data set and a 91.97% accuracy on the testing data set. [ABSTRACT FROM AUTHOR]

This paper represents a constraint planning and optimization control scheme for a highly redundant mobile manipulator considering a complex indoor environment. Compared with the traditional optimization solution of a redundant manipulator, infinity norm and slack variable are additionally introduced and leveraged by the optimization algorithm. The former takes into account the joint limits effectively by considering individual joint velocities and the latter relaxes the equality constraint by decreasing the infeasible solution area. By using derived kinematic equations, the tracking control problem is expressed as an optimization problem and converted into a new quadratic programming (QP) problem. To address the optimization problem, the two-timescale recurrent neural networks optimization scheme is proposed and tested with a 9 DOFs nonholonomic mobile-based manipulator. Additionally, the B I 2 R R T ∗ path-planning algorithm incorporates path planning in the complex environment where different obstacles are positioned. To test and evaluate the proposed optimization scheme, both predefined and generated paths are tested in the Neurorobotics Platform (NRP) 2 2 https://neurorobotics.net. which is open access and open source integrative simulation framework powered by Gazebo and developed by our team. • The kinematic model of a 9 DOFs mobile-based manipulator is derived. • The tracking control problem with infinity norm and slack variable is formulated. • A two-timescale recurrent neural (TNN) networks optimization scheme is introduced • TNN addresses the hierarchical optimization problem with planning trajectory. • The experiments using B I 2 R R T ∗ are demonstrated in Neurorobotics platform. [ABSTRACT FROM AUTHOR]

Zeroing neural network (ZNN) model, an important class of recurrent neural network, has been widely applied in the field of computation and optimization. In this paper, two ZNN models with predefined-time convergence are proposed for the time-varying quadratic programming (TVQP) problem. First, in the framework of the traditional ZNN model, the first-order predefined-time convergent ZNN (FPTZNN) model is proposed in combination with a predefined-time controller. Compared with the existing ZNN models, the proposed ZNN model is error vector combined with sliding mode control technique. Then, the FPTZNN model is further extended and the second-order predefined-time convergent ZNN (SPTZNN) model is developed. Combined with the Lyapunov method and the concept of predefined-time stability, it is shown that the proposed FPTZNN and SPTZNN models have the properties of predefined-time convergence, and their convergence time can be flexibly adjusted by predefined-time control parameters. Finally, the proposed FPTZNN and SPTZNN models are compared with the existing ZNN models for the TVQP problem in simulation experiment, and the simulation experiment results verify the effectiveness and superior performance of the proposed FPTZNN and SPTZNN models. In addition, the proposed FPTZNN model for robot motion planning problem is applied and successfully implemented to verify the practicality of the model. • A first-order neural network model is established. • Further, a second-order neural network model is established. • Experiments with numerical example and robot motion planning problem. [ABSTRACT FROM AUTHOR]

Nowadays, meteorological data plays a crucial role in various fields such as remote sensing, weather forecasting, climate change, and agriculture. The regional and local studies call for high spatial resolution gridded meteorological data to identify refined details, which however is generally limited due to the models, platforms, sensors, etc. Downscaling has been a significant and practical way to improve spatial resolution. In recent years, with superior feature extraction and expression abilities, deep learning (DL) has outperformed traditional methods in various areas, and exhibits huge potential to establish a complicated mapping between large-scale and local-scale meteorological data. Therefore, this paper provides a systematic review of DL in statistical downscaling for deriving high spatial resolution gridded meteorological data. This review first presents the background, including a taxonomy of downscaling methods, the role of DL in statistical downscaling, and the analogy between downscaling and image super-resolution. It shows evidence of how downscaling can benefit from DL, particularly super-resolution networks. Subsequently, this review focuses on the recent development of the DL-based statistical downscaling of gridded meteorological data, especially the deep architectures, including convolutional neural networks to capture the spatial dependencies of meteorological variables, recurrent neural networks to reveal the temporal states from time series, and generative adversarial networks to facilitate the reconstruction of high-frequency details, as well as the major structure residual learning and attention mechanism. In addition, this review demonstrates the specific issues towards downscaling, including scaling factors, spatial–temporal and variable correlations, and paired datasets construction, and then gives a comprehensive summary of the status of datasets, toolsets and metrics. The future challenges from the perspective of unsupervised models, transformer architecture, data fusion, physical-informed learning, generalization capacity, and uncertainty quantification for downscaling are finally discussed. • Deep learning maps large-scale predictors to local-scale predictand for downscaling. • The major deep structures for downscaling networks and enhancement are reviewed. • The specific issues on downscaling factor, correlations, and datasets are addressed. • Unsupervised, physical-informed learning, uncertainty etc. pose great challenges. [ABSTRACT FROM AUTHOR]

This paper addresses the dynamic quaternion-valued Sylvester equation (DQSE) using the quaternion real representation and the neural network method. To transform the Sylvester equation in the quaternion field into an equivalent equation in the real field, three different real representation modes for the quaternion are adopted by considering the non-commutativity of quaternion multiplication. Based on the equivalent Sylvester equation in the real field, a novel recurrent neural network model with an integral design formula is proposed to solve the DQSE. The proposed model, referred to as the fixed-time error-monitoring neural network (FTEMNN), achieves fixed-time convergence through the action of a state-of-the-art nonlinear activation function. The fixed-time convergence of the FTEMNN model is theoretically analyzed. Two examples are presented to verify the performance of the FTEMNN model with a specific focus on fixed-time convergence. Furthermore, the chattering phenomenon of the FTEMNN model is discussed, and a saturation function scheme is designed. Finally, the practical value of the FTEMNN model is demonstrated through its application to image fusion denoising. [ABSTRACT FROM AUTHOR]

The soft manipulator actuated parallel with contractile and extensor pneumatic artificial muscles (PAMs) can perform continuous operations and vary stiffness. Due to the hysteresis nonlinearity of PAMs and parallel driving, the manipulator shows coupled hysteresis with stiffness-dependent and rate-dependent characteristics, causing inaccurate positioning performance. This paper firstly proposes a comprehensive dynamic model consisting of two parts in series to predict the manipulator dynamical behaviors with coupled hysteresis. Furthermore, a decoupled inverse compensation and decoupling control on the basis of the proposed model is conducted on the soft manipulator. Experimental results and comparisons illustrate the high precision and strong generalization ability of the proposed model and the validity of the compensator in mitigating the coupled dynamic hysteresis of the system. • A coupled dynamic hysteresis model is proposed for the PAM-based manipulator system. • The model is based on UPI operators and the fuzzy recurrent neural network. • An inverse compensation control is developed to decouple and compensate the system. • Validations of the proposed model and compensation method are conducted. [ABSTRACT FROM AUTHOR]

• Processing Complex-valued PolSAR Data Using Complex-valued Convolutional Neural Network (CV-CNN). • Uses a Gaussian-type activation function (GTAF) that preserves the integrity of complex-valued operations. • Introduces learnable Gaussian parameters for GTAF, and designs two multi-channel activation methods. • The classification accuracy is better than that of existing state-of-the-art methods in three datasets. To process complex-valued information such as SAR signals conveniently, the complex-valued convolutional neural network (CV-CNN) has been proposed in recent years, and it has achieved great success in SAR image recognition. This paper proposes an activation function with learnable parameters based on the Gaussian-type activation function (GTAF) in CV-CNN to improve the utilization of information in the real and imaginary parts of the neuro. For the multi-channel input of the feature map, this paper discusses two ways to set the parameters of the Gaussian-type activation function. One is that all channels share the same parameters, called the channel-sharing Gaussian-type activation function (CSGTAF). The other is that each channel has its independent parameters, called the channel-exclusive Gaussian-type activation function (CEGTAF). In addition, this paper derives the backpropagation formula of both CSGTAF and CEGTAF in detail for the training process of CV-CNN. This paper performs experimental analysis on three L-band standard PolSAR datasets. The experimental results show that, compared with the traditional method and the Gaussian activation function with fixed parameters, both CSGTAF and CEGTAF achieve higher recognition accuracy, and the difference in the recognition effect of different targets in the same dataset is little. Both show good recognition performance and have good stability and versatility. [ABSTRACT FROM AUTHOR]

Identification of human actions from video has gathered much attention in past few years. Most of the computer vision tasks such as Health Care Activity Detection, Suspicious Activity detection, Human Computer Interactions etc. are based on the principle of activity detection. Automatic labelling of activity from videos frames is known as activity detection. With the introduction of deep networks, the process of activity detection is clustered into two groups known as hand-crafted feature based approach and automatic feature extraction approach. This paper focuses on various approaches used in recent literature based on traditional and automatic approach. Moreover, hierarchy for different approaches under them such as space based, motion based, genetic based, fuzzy based, dictionary based are discussed. With introduction of Convolutional Neural Networks and Recurrent Neural Networks, automatic learning capability from input modality makes them first choice to be implemented for activity recognition. In this paper various approaches have been analyzed according to methodology, accuracy, classifier and datasets. [ABSTRACT FROM AUTHOR]

This paper focuses on the application of meta-reinforcement learning to the robust design of low-thrust interplanetary trajectories in the presence of multiple uncertainties. A closed-loop control policy is used to optimally steer the spacecraft to a final target state despite the considered perturbations. The control policy is approximated by a deep recurrent neural network, trained by policy-gradient reinforcement learning on a collection of environments featuring mixed sources of uncertainty, namely dynamic uncertainty and control execution errors. The recurrent network is able to build an internal representation of the distribution of environments, thus better adapting the control to the different stochastic scenarios. The results in terms of optimality, constraint handling, and robustness on a fuel-optimal low-thrust transfer between Earth and Mars are compared with those obtained via a traditional reinforcement learning approach based on a feed-forward neural network. • In interplanetary space missions, the spacecraft trajectory is affected by multiple uncertainties. • Meta-reinforcement learning can be applied seamlessly to any uncertainty and dynamic model. • A recurrent neural network is used as a history-dependent closed-loop control policy. • The network is trained on a low-thrust transfer featuring dynamic uncertainties and control execution errors. • Meta-reinforcement learning shows improved performance and robustness compared to standard reinforcement learning. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new short-term load forecasting (STLF) model based on contextually enhanced hybrid and hierarchical architecture combining exponential smoothing (ES) and a recurrent neural network (RNN). The model is composed of two simultaneously trained tracks: the context track and the main track. The context track introduces additional information to the main track. It is extracted from representative series and dynamically modulated to adjust to the individual series forecasted by the main track. The RNN architecture consists of multiple recurrent layers stacked with hierarchical dilations and equipped with recently proposed attentive dilated recurrent cells. These cells enable the model to capture short-term, long-term and seasonal dependencies across time series as well as to weight dynamically the input information. The model produces both point forecasts and predictive intervals. The experimental part of the work performed on 35 forecasting problems shows that the proposed model outperforms in terms of accuracy its predecessor as well as standard statistical models and state-of-the-art machine learning models. [ABSTRACT FROM AUTHOR]

In this paper, we propose an event-triggered collaborative neurodynamic approach to distributed global optimization in the presence of nonconvexity. We design a projection neural network group consisting of multiple projection neural networks coupled via a communication network. We prove the convergence of the projection neural network group to Karush–Kuhn–Tucker points of a given global optimization problem. To reduce communication bandwidth consumption, we adopt an event-triggered mechanism to liaise with other neural networks in the group with the Zeno behavior being precluded. We employ multiple projection neural network groups for scattered searches and re-initialize their states using a meta-heuristic rule in the collaborative neurodynamic optimization framework. In addition, we apply the collaborative neurodynamic approach for distributed optimal chiller loading in a heating, ventilation, and air conditioning system. [ABSTRACT FROM AUTHOR]

Energy generated from renewable sources is exposed to extremely dynamic variations in climatic conditions as well as uncertainties (current/voltage variability, noise, measurement errors.). These conditions are relevant issues to be considered when monitoring renewable energy conversion (REC) systems. In fact, these uncertain systems are subjected to many failures leading to performance degradation and long downtime maintenance periods. Therefore, fault detection and diagnosis (FDD) are essential to ensure its high dependability. This paper proposes an FDD under climatic conditions variability of uncertain REC systems using deep recurrent neural networks (DRNNs) techniques. Firstly, a novel modeling strategy for REC systems is built. Secondly, different DRNN-based interval-valued data methods are intended to differentiate between the various REC systems operating states. Finally, the hyperparameters of the proposed techniques are tuned using the Bayesian optimization algorithm. The efficiency and robustness of the novel strategy are demonstrated through REC application, using grid-connected photovoltaic (GCPV) systems. The obtained results show the efficiency of the developed strategy by reaching an accuracy rate of 92.40%. [ABSTRACT FROM AUTHOR]

To meet productivity goals, machining operations require high levels of efficiency to achieve the desired manufacturing quality in the available time. Additionally, machine tools can lead to safety-critical scenarios due to the high process dynamics and hazardous forces and torques. Therefore, workers need to exhibit special care while interacting with certain kinds of machines. Operational safety can be increased by eradicating the dependence on tangible control of machine tools. Current developments in Artificial Intelligence, especially in natural language processing, lead to increased utilization of speech-based interaction between humans and machines. However, these developments are mainly limited to areas of consumer electronics. Nevertheless, this type of human-machine interaction offers great potential for machining operations. Artificial neural network architectures like Transformer have rapidly emerged as the dominant architecture for natural language processing, overtaking alternative neural architectures such as Convolutional and Recurrent Neural Networks in performance for natural language understanding and natural language generation tasks. The architecture scales with training data and model size, enables efficient parallel training, and captures wide-ranging sequence features. These advancements can be used for enhancing the ease of use and safety of machine tools, which are often safety critical. In this article, a Voice-User-Interface based control of machine tools is conceptually outlined. Such a control can increase the efficiency of the workers, as it eradicates the need of hand usage. It further increases the safety of workers operating the machines and can detect anomalous sounds corresponding to machine failures during the machining process. The voice is first passed through a speech-to-text pipeline for converting the speech signals to relevant text, the text output is mapped to machine commands and passed to the physical machine. The implementation of the setup guarantees the speech-based control – even for safety-critical tasks -of machine tools. Moreover, this paper outlines the benefits and challenges of the developed VUI architecture. [ABSTRACT FROM AUTHOR]

This paper conducts comparative analysis of speaker identification techniques, namely Gaussian Mixture Models (GMM), Bidirectional Long Short-Term Memory (BiLSTM) neural networks, and Recurrent Neural Networks (RNN), applied to the Kazakh Speech Corpus. The study's central objective is to explore the influence of speech duration on the efficacy of speaker identification. Our results indicate that all models exhibit enhanced accuracy with increased speech duration. Furthermore, the incorporation of delta and fundamental frequency (F0) as supplementary features bolsters the performance of these models. This research contributes to the understanding of how speech duration and additional features can optimize speaker identification techniques, providing valuable insights for future developments in the field. [ABSTRACT FROM AUTHOR]

Digital Twins have much attention in the shipping industry, attempting to support all phases of a vessel's life cycle. With several tools appearing in Digital Twin software suites, high-quality manoeuvring and performance prediction remain cornerstones. Propulsion efficiency is in focus while in service. Simulator-based training is in focus to ensure safety of manoeuvring in confined waters and harbours. Prediction of ships' velocity and turn rate are essential for correct look and feel during training, but phenomena like dynamic inflow to propellers, bank and shallow water effects limit simulators' accuracy, and master mariners often comment that simulations could be in better agreement with actual behaviours of their vessel. This paper focuses on digital twin enhancements to better match reality. Using data logged during in-service operation, we first consider a system identification perspective, employing a first-principles model structure. Showing that a complete first-principles model is not identifiable under the excitation met in service, we employ a Recurrent Neural Network to predict deviations between measured velocities and the model output. The outcome is a hybrid of a first-principles model with a machine learning generic approximator add-on. The paper demonstrates significant improvements in prediction accuracy of both in-harbour manoeuvring and shallow water passage conditions. • Identifiability analysis to highlight challenges in identification of first-principle hydrodynamics models. • Hybrid model for velocity prediction that combines a first-principles and Recurrent Neural Networks. • Training and validation using full-scale data collected from an operational Roll-ON/Roll-Off ferry. • Systematic evaluation of the obtained model accuracy and comparison to an industrial state-of-the-art ship simulator. [ABSTRACT FROM AUTHOR]

Sentiment analysis aims to classify a text according to sentimental polarities of people's opinions, such as positive, negative, or neutral. While most of the studies focus on eliciting features from English text, the research on Arabic is limited due to the morphological and grammatical complexity of Arabic language. In this paper, we proposed an optimized sentiment classification for dialectal Arabic short text at the document level using deep learning (DL). The contributions of this paper are in three areas. First, we extracted semantic features for Arabic short text at the word level and character level. Second, we used three DL topologies for classification models: a long short-term memory recurrent neural network (LSTM); a convolutional neural network (CNN); and an ensemble model combining both models' advantages to improve the prediction performance. Third, we used a hyperparameter tuning estimation method to optimize the neural network performance. We trained and tested our proposed models on a dataset that consists of Modern Standard Arabic and dialectal Arabic corpus collected from Twitter. The results showed significant improvement in Arabic text classification in term of classification accuracy that ranges between 88% and 69.7%. The ensemble model scored the highest accuracy of 96.7% on the test set. [ABSTRACT FROM AUTHOR]

With the increasing influence of new energy power system, the prediction of Photovoltaic (PV) output power becomes more and more important In this paper, it is the first time to put forward a hybrid modeling method combining long–short term memory recurrent neural network (LSTM) and stochastic differential equation (SDE). This method realizes the prediction of PV output power in different seasons and overcomes the uncertainty of PV power generation. Wavelet analysis and automatic encoder are used to decompose data and extract important features. According to the detailed signal sequence and the approximate signal sequence, the LSTM prediction model is established. Meanwhile, the mathematical model of SDE is established according to the detailed signal sequence. Finally, the output sequences of the two models are reconstructed by wavelet transform. This hybrid model can not only realize the point prediction of PV output power according to the predicted mean value, but also achieve the interval prediction under different confidence levels according to the randomness. In this paper, the proposed method is applied to predict the PV output power of CHINT photovoltaic power generation system with installed capacity of 10MW in different seasons, and the weather forecast data with errors of ± 10%, ± 20% and ± 30% are used. Experimental results prove the effectiveness of the method. In the summer model considering forecast errors within ± 20% of weather forecast data, the RMSEs of BP neural network, LSTM and convolutional neural network (CNN) are 5.9468, 5.6762 and 5.8004 respectively. However, the RMSE of the mean prediction with the confidence level of 90% under the proposed method is 4.4647. With this method, the results of interval prediction and point prediction of PV output power can provide better decision support for the stable and safe operation of PV grid connection. They have higher reference value for energy dispatching departments. • Combining LSTM neural network with stochastic differential equation for the first time by wavelet analysis. • The features of weather data are strongly extracted by autoencoder. • Application of hybrid modeling method to photovoltaic power prediction. • Obtaining different confidence intervals by changing random factors. [ABSTRACT FROM AUTHOR]

This paper delivers an exhaustive analysis of the fusion of multi-sensor technologies, including traditional sensors such as cameras, Light Detection and Ranging(LiDAR), Radio Detection and Ranging(RADAR), and ultrasonic sensors, with Artificial Intelligence(AI) powered methodologies in obstacle detection for Autonomous Vehicles(AVs). With the growing momentum in AVs adoption, a heightened need exists for versatile and resilient obstacle detection systems. Our research delves into study of literatures, where proposed approaches assimilate data from this diverse sensor suite, integrated through Deep Learning(DL) techniques, to refine AV performance. Recent advancements and prevailing challenges within the domain are thoroughly examined, with particular focus on the integration of sensor fusion techniques, the facilitation of real-time processing via edge and fog computing, and the implementation of advanced artificial intelligence architectures, including Convolutional Neural Networks(CNNs), Recurrent Neural Networks(RNNs), and Generative Adversarial Networks(GANs), to enhance data interpretation efficacy. In conclusion, the paper underscores the critical contribution of multi-sensor arrays and deep learning in enhancing the safety and reliability of autonomous vehicles, offering significant perspectives for future research and technological progress. [ABSTRACT FROM AUTHOR]

Current failure prediction methods of bearings have less uncertainty analysis with interpretability, less correlation analysis between degradation characteristics and prediction error. Moreover, there are multiple degradation stages in entire life cycle and prediction performance cannot meet practical demands. Therefore, this paper proposes a new approach for failure prediction of bearings. The change point detection method achieves multi-stage division of degradation data. The improved hybrid deep forest with best dissimilarity sequence (BDS) is studied and a new pretrained algorithm with pruning operation is developed. The convergence theorem is proved. A novel multi-stage failure prediction algorithm based on improved hybrid deep forest, hypothesis testing and interpretability analysis, is developed to get better prediction result. The time complexity of proposed algorithm is analyzed. The datasets of NASA and FEMTO-ST institute are utilized and experimental results show that: 1) Our approach with model interpretability has better prediction performances than support vector machine (SVR), recurrent neural network (RNN), long short-term memory (LSTM), and deep forest (DF); 2) The non-normal distribution characteristics, monotonic degradation trend and effect size of multiple stages are analyzed based on hypothesis testing methods; 3) The positive and inverse relation analysis achieves the correlation interpretability between multi-stage degradation characteristics and failure prediction results. • EWMA, CUSUM, and K-means clustering are used to obtain real change points for multi-stage division. • An improved hybrid deep forest model with BDS is presented to improve prediction performance. • This paper proposes a new pretrained algorithm to achieve a better tradeoff between accuracy and runtime cost. • A multi-stage failure prediction algorithm with model interpretability is developed. • Hypothesis testing and correlation analysis are utilized to enhance the interpretability of degradation characteristics. [ABSTRACT FROM AUTHOR]

Pipe wall-thinning is a phenomenon whereby the thickness of pipes in a nuclear power plant decreases over time owing to extended operational years. This thickness reduction is caused by various long-term thermal aging mechanisms such as Flow-Accelerated Corrosion, Liquid Droplet Impingement Erosion, cavitation, and flashing. Reducing the thickness of the secondary system pipes to the point of rupture can lead to severe human casualties and significant economic losses. Consequently, domestic power plant operators regularly manage the power plant pipes. Ongoing research focuses on the identification and management of pipe wall-thinning. However, previous studies have encountered problems in accurately judging the decrease in pipe wall-thinning in the presence of noise. To overcome this, Convolutional Neural Network (CNN) models based on image feature analysis have been used. This approach allows for the differentiation of pipe wall-thinning feature from small-sized noise in a single image. However, there were difficulties in making accurate judgments for large-sized noise that resembled the feature of pipe wall-thinning. This paper aims to analyze the limitations of the current pipe wall-thinning evaluation methods and to achieve accurate pipe wall-thinning discrimination through time-series analysis of continuous pipe wall-thinning data. The proposed method employs a Convolutional Recurrent Neural Network (CRNN) model, integrating the Recurrent Neural Network(RNN) model with the CNN model. The image feature of the pipes, extracted using CNN, are utilized as inputs for the RNN. This enables the observation of how the image features of the pipes change over time. This feature differentiates from the time-series feature of noise that occurs suddenly. Through this method, the paper proposes a new approach for effectively identifying the gradual decrease in pipe wall-thinning, enabling precise assessment of pipe wall-thinning progression. [ABSTRACT FROM AUTHOR]

Over the past decades, deep learning techniques have attracted increased attention from various research and industrial domains aligned with the development of Industry Internet-of-Things(IIoT). Specifically, with the advantage of data-driven methods, industrial organizations are seeking novel proactive strategies supported by analytic models to guarantee the quality of their production by observing degradation or predicting failure ahead of the occurrence of the component or asset. Predictive strategies are expected to promise the influence of unnecessary maintenance interruptions and mitigate the consequence of that, hence, extending the remaining useful life of products. This paper conducts a survey of the utilization of deep learning technologies on engineering applications where they provide satisfactory solutions with respect to specific data types or input signals. 106 primary papers are reviewed on deep learning–driven approaches which mainly explore five of the most popular architectures in the application of predictive maintenance. The main content of this paper summarizes the common advantages of each architecture and, accordingly, points out their limitations, as well as describes the application scopes of fully connected deep neural networks, convolutional neural networks, stacked autoencoders, deep belief networks, and deep recurrent neural networks. Based on the technique discussion for each of them, we intend to provide a comprehensive understanding and guidance of the appropriate usage of deep learning architectures to devise an effective predictive maintenance strategy for the scientific and industrial developers whose expertise lies in the prior domain knowledge of multi-source isomerization data. Moreover, the main content demonstrated the summarization of the decisive factor by which the incremental stages of the approaches were determined, fundamentally including the dataset specification, feature extraction, and the integration of deep learning approaches. [ABSTRACT FROM AUTHOR]

The paper presents the application of neural networks evolving under the Hill Climb Assembler Encoding (HCAE) algorithm to control follower autonomous underwater vehicles that are members of a swarm consisting of one leader vehicle and a group of followers. The leader is responsible for global navigation and guiding the swarm, while low-cost followers unable to cover longer distances on their own follow the leader. To locate the leader, the followers only use information about the distance to it. Directional information is unavailable to the followers. Moreover, information about the distance is transmitted to followers with a frequency depending on the number of followers: the more followers, the lower the frequency. In addition to tracking the leader, the followers must also be able to avoid collisions with other followers and the leader. To this end, they are equipped with short-range sensors looking around each follower. The simulations presented in the paper were carried out for different swarm sizes, different sensor ranges, different collision distances, and variable leader speed, and showed high effectiveness of the proposed neural solution. The most common strategy for the followers when following a leader was to circle it at a certain safe distance depending on the sensor range. • Neuro-evolutionary controller for a swarm of autonomous underwater vehicles is proposed. • The task of the controller is to keep the swarm in one compact group while moving along a predefined trajectory. • The only information provided to the controller is the distance to the swarm leader, directional information is unavailable. [ABSTRACT FROM AUTHOR]

Stereo matching acts a crucial role in computer vision and robotics applications. An accurate cost volume and robust disparity regression method are essential for stereo matching of high accuracy. Following GCNet and PSMNet, constructing 4D cost volume and then using the soft argmin method to regress has been dominated. However, it will encounter many difficulties due to the multi-modal distribution of cost volume. One of the reasons for this multi-modal distribution is the occlusion area which not be possible to find a matching region on the reference image and rarely discussed. In this paper, we propose to use global context information could improve the performance of model in occluded regions. Recently, novel recurrent neural network regression methods are proposed, but most of them regress disparity maps from 3D cost volume. In this paper, we propose the new combinatorial paradigm that combine stacked hourglass modules and recurrent neural networks to further aggregate 4D cost volume and regress disparity respectively. The proposed method can be seamlessly integrated into most stereo matching networks, we improved the accuracy by 45% for PSMNet and 38% for GwcNet in our experiment. Experimental results on Scene Flow, KITTI2012, KITTI2015, and ETH3D datasets show our method is competitive. The code is available at: https://github.com/truman1211/HCRnet. • Replacing the soft argmin method with GRU modules. • Different context information is embedded in different stages of the GRU. • The GRU model is regressed on the cost volume at different scales. • Attention mechanism is used to provide global information for the occlusion region. [ABSTRACT FROM AUTHOR]

This article has been retracted: please see Elsevier Policy on Article Withdrawal (https://www.elsevier.com/locate/withdrawalpolicy). This article has been retracted at the request of the Editor-in-Chief. Following peer review and publication of this paper, the editors were made aware that this paper had been submitted simultaneously elsewhere and ultimately published. This paper is a duplicate submission of the following: https://link.springer.com/article/10.1007/s11760-024-03062-y. The timing of submission and publication was such that the duplication remained undetected during peer review. The editors apologise for any inconvenience caused to readers. Redundant publications overweigh the relative importance of published findings and distort the academic record of the authors. One of the conditions of submission of a paper for publication is therefore that authors declare explicitly that the paper has not been previously published and is not under consideration for publication elsewhere. As such this article represents a misuse of the scientific publishing system. The scientific community takes a very strong view of this matter and apologies are offered to readers of the journal that this was not detected during the submission process. [ABSTRACT FROM AUTHOR]

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals. [ABSTRACT FROM AUTHOR]

Motion matching has become a widely adopted technique for generating high-quality interactive animation systems in video games. However, its current implementations suffer from significant computational and memory resource overheads, limiting its scalability in the context of modern video game performance profiles. Our method significantly reduces the computational complexity of approaches to motion synthesis, such as "Learned Motion Matching", while simultaneously improving the compactness of the data that can be stored and the robustness of pose output. As a result, our method enables the efficient execution of motion matching that significantly outperforms other implementations, by 8.5× times in CPU execution cost and at 80% of the memory requirements of "Learned Motion Matching", on contemporary video game hardware, thereby enhancing its practical applicability and scalability in the gaming industry and unlocking the ability to apply on large numbers of animated in-game characters. In this paper, we expand upon our published paper "Learning Robust and Scalable Motion with Lipschitz Continuity and Sparse Mixture of Experts", where we successfully proposed a novel method for learning motion matching that combines a Sparse Mixture of Experts model architecture and a Lipschitz-continuous latent space for representation of poses. We present further details on our method, with extensions to our approach to expert utilization within our neural networks. [Display omitted] • Applies the Sparse Mixture of Experts architecture to animation synthesis for the first time. • Significantly reduce theoretical and practical compute requirements, 8.5 times that of prior art. • Lipschitz-continuous encoding neural networks producing a linearly interpolatable manifold. • SMoE Networks express a higher density of animation data than Feed-Forward Neural Networks. [ABSTRACT FROM AUTHOR]

• Terahertz time-domain spectroscopy contains noises and uncertainties. • The noises and uncertainties add difficulty to spectral signal analysis. • A recurrent quantum neural network was proposed as an adaptive filter. • The proposed network can eliminate the noise and maintain spectrum characteristics. System interferences such as noise and background information in terahertz time-domain spectroscopy (THz-TDS) add difficulty to spectral signal analysis and affect the accuracy of component identification. In this paper, we proposed an adaptive filter based on a recurrent quantum neural network (RQNN) to process the THz-TDS signal. RQNN is a network architecture guided by quantum mechanics, for adaptive filtering of THz-TDS signal. Six pure substances, ferric sulfate, ferrous sulfate, Prussian blue, agarose, polylysine and sodium of polyaspartic acid were used as test samples. The TDS signals of the six samples were filtered with RQNN and five competing filters. Optical constants including refractive index and absorption coefficient were calculated from raw THz signal and filtered THz signal. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were calculated between the optical constants calculated before and after filtering as evaluation indicators. Results revealed that RQNN could better eliminate the noise in THz-TDS signals, and retain the characteristic peaks in the calculated optical constants simultaneously. In terms of PSNR and SSIM, RQNN-filtered THz-TDS signals demonstrated higher values than those of the competing filters. The RQNN proposed in this paper can be used as an adaptive filter for noise filtering and signal extraction of THz-TDS signal, with advantages above traditional filtering algorithms. [ABSTRACT FROM AUTHOR]

Transformers are Deep Neural Networks (DNN) that utilize a self-attention mechanism to capture contextual relationships within sequential data. Unlike traditional neural networks and variants of Recurrent Neural Networks (RNNs), such as Long Short-Term Memory (LSTM), Transformer models excel at managing long dependencies among input sequence elements and facilitate parallel processing. Consequently, Transformer-based models have garnered significant attention from researchers in the field of artificial intelligence. This is due to their tremendous potential and impressive accomplishments, which extend beyond Natural Language Processing (NLP) tasks to encompass various domains, including Computer Vision (CV), audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published, spotlighting the Transformer's contributions in specific fields, architectural disparities, or performance assessments, there remains a notable absence of a comprehensive survey paper that encompasses its major applications across diverse domains. Therefore, this paper addresses this gap by conducting an extensive survey of proposed Transformer models spanning from 2017 to 2022. Our survey encompasses the identification of the top five application domains for Transformer-based models, namely: NLP, CV, multi-modality, audio and speech processing, and signal processing. We analyze the influence of highly impactful Transformer-based models within these domains and subsequently categorize them according to their respective tasks, employing a novel taxonomy. Our primary objective is to illuminate the existing potential and future prospects of Transformers for researchers who are passionate about this area, thereby contributing to a more comprehensive understanding of this groundbreaking technology. • The paper presents a comprehensive survey on transformers for deep learning tasks. • The paper conducts a thorough analysis on highly effective models in five domains. • The paper classifies the models based on respective tasks using a proposed taxonomy. • The characteristics of the surveyed models are deeply explored and analyzed. • Future directions and challenges for transformer-based models are deciphered. [ABSTRACT FROM AUTHOR]

Weather forecasting is the practice of predicting the state of the atmosphere for a given location based on different weather parameters. Weather forecasts are made by gathering data about the current state of the atmosphere. Accurate weather forecasting has proven to be a challenging task for meteorologists and researchers. Weather information is essential in every facet of life like agriculture, tourism, airport system, mining industry, and power generation. Weather forecasting has now entered the era of Big Data due to the advancement of climate observing systems like satellite meteorological observation and also because of the fast boom in the volume of weather data. So, the traditional computational intelligence models are not adequate to predict the weather accurately. Hence, deep learning-based techniques are employed to process massive datasets that can learn and make predictions more effectively based on past data. The effective implementation of deep learning in various domains has motivated its use in weather forecasting and is a significant development for the weather industry. This paper provides a thorough review of different weather forecasting approaches, along with some publicly available datasets. This paper delivers a precise classification of weather forecasting models and discusses potential future research directions in this area. [ABSTRACT FROM AUTHOR]

This paper aims to discuss and analyze the potentialities of Recurrent Neural Networks (RNN) in control design applications. The main families of RNN are considered, namely Neural Nonlinear AutoRegressive eXogenous, Echo State Networks, Long Short Term Memory, and Gated Recurrent Units. The goal is twofold. Firstly, to survey recent results concerning the training of RNN that enjoy Input-to-State Stability (ISS) and Incremental Input-to-State Stability (δ ISS) guarantees. Secondly, to discuss the issues that still hinder the widespread use of RNN for control, namely their robustness, verifiability, and interpretability. The former properties are related to the so-called generalization capabilities of the networks, i.e. their consistency with the underlying real plants, even in presence of unseen or perturbed input trajectories. The latter is instead related to the possibility of providing a clear formal connection between the RNN model and the plant. In this context, we illustrate how ISS and δ ISS represent a significant step towards the robustness and verifiability of the RNN models, while the requirement of interpretability paves the way to the use of physics-based networks. The design of model predictive controllers with RNN as plant's model is also briefly discussed. Lastly, some of the main topics of the paper are illustrated on a simulated chemical system. [ABSTRACT FROM AUTHOR]

Deep learning methods possess many processing layers to understand the stratified representation of data and have achieved state-of-art results in several domains. Recently, deep learning model designs and architectures have unfolded in the context of Natural Language Processing (NLP). This survey presents a brief description of the advances that have occurred in the area of Deep Generative modeling. This work considers most of the papers from 2015 onwards. In this paper, we review many deep learning models that have been used for the generation of text. We also summarize the various models and have put forward the detailed understanding of past, present, and future of text generation models in deep learning. Furthermore, DL approaches that have been explored and evaluated in different application domains in NLP are included in this survey. [ABSTRACT FROM AUTHOR]

Multitask learning (MTL) aims at beneficial joint solving of multiple prediction problems by sharing information across different tasks. However, without adequate consideration of interdependencies, MTL models are prone to miss valuable information. In this paper, we introduce a novel deep MTL architecture that specifically encodes cross-task interdependencies within the setting of multiple image classification problems. Based on task-wise interim class label probability predictions by an intermediately supervised hard parameter sharing convolutional neural network, interdependencies are inferred in two ways: i) by directly stacking label probability sequences to the image feature vector (i.e. , multitask stacking), and ii) by passing probability sequences to gated recurrent unit-based recurrent neural networks to explicitly learn cross-task interdependency representations and stacking those to the image feature vector (i.e. , interdependency representation learning). The proposed MTL architecture is applied as a tool for generic multi-criteria building characterization using street-level imagery related to risk assessments toward multiple natural hazards. Experimental results for classifying buildings according to five vulnerability-related target variables (i.e., five learning tasks), namely height , lateral load-resisting system material , seismic building structural type , roof shape, and block position are obtained for the Chilean capital Santiago de Chile. Our MTL methods with cross-task label interdependency modeling consistently outperform single task learning (STL) and classical hard parameter sharing MTL alike. Even when starting already from high classification accuracy levels, estimated generalization capabilities can be further improved by considerable margins of accumulated task-specific residuals beyond +6% κ. Thereby, the combination of multitask stacking and interdependency representation learning attains the highest accuracy estimates for the addressed task and data setting (up to cross-task accuracy mean values of 88.43% overall accuracy and 84.49% κ). From an efficiency perspective, the proposed MTL methods turn out to be substantially favorable compared to STL in terms of training time consumption. [ABSTRACT FROM AUTHOR]

Estimating the remaining useful life (RUL) of proton exchange membrane fuel cells (PEMFCs) is key to predicting and increasing their durability. This paper investigates six deep learning techniques – long short-term memory (LSTM) networks, gated recurrent units (GRU), 1-D convolutional neural network (CNN)-LSTMs, 1D–CNN–GRUs, 1D–CNN–Bidirectional-LSTMs, and 1D–CNN–Bidirectional-GRUs – for RUL and long-term degradation trend prediction for a five-cell PEMFC stack operated with and without current ripples. The hyperparameters of all models are optimized for minimum root-mean-square error (RMSE) of the stack voltage using tree-structured Parzen estimators (TPE) with hyperband pruning. The obtained prediction results are benchmarked using the coefficient of determination (R2), RMSE, and mean absolute percentage error (MAPE) in stack voltage predictions and relative error (RE) in RUL estimates. The comparative analysis for models trained on 50% datasets reveals that the best R2, RMSE, MAPE, and RE of 98.87E-2, 1.258E-3, 0.285E-1, and 0.0425E-1 are offered by the LSTM model for the stack without current ripples. Further, for the stack with current ripples, the best R2, RMSE, MAPE, and RE of 99.482E-2, 1.77E-3, 0.444E-1, and 0.089E-1 are exhibited by the 1D–CNN–Bidirectional-GRU model. The LSTM model also reveals excellent transfer learning performance on the stack with current ripples with the best R2, RMSE, MAPE, and RE of 98.91E-2, 2.455E-3, 0.362E-1, and 0.044E-1, respectively. [Display omitted] • Six deep learning models are developed for PEMFC RUL and degradation prediction. • Hyperparameters of each model are optimized using TPE for static and dynamic loading. • Static loading models exhibited excellent transfer learning performance. • All tuned models performed superior to recent reported approaches in RMSE. • LSTM and 1D–CNN–Bidirectional-GRU demonstrated best results trained on 50% data. [ABSTRACT FROM AUTHOR]

Plant diseases can significantly reduce food and agricultural production, leading to major quality, quantity, and economic losses. Plant disease deficits are usually reduced by early diagnosis through visual observation. Significant plant species grown in specific parts of the world include olives. Depending on the place where an olive tree is produced, many diseases can affect it. Traditional plant disease detection is ineffective and time-consuming. Therefore, this paper developed an in-depth evaluation of RNN architecture with an Ant Colony Optimization algorithm for disease identification in olive trees. It consists of a dataset that contains 3300 images of healthy and diseased leaves. The images are gathered in a dataset and then pro-processed. After pre-processing the images, the segmentation process is done using Wavelet transform. Then, the Ant Colony Optimization algorithm is employed for extracting the features. Lastly, categorization is done by applying RNN. The results suggest the optimal model for creating an efficient disease detector. The developed method attains higher performance when compared with other existing techniques. Hence, the outcomes demonstrate that the ACO-RNN technique has a reliable potential for identifying plant infections. [ABSTRACT FROM AUTHOR]

This paper introduces a recurrent general type-2 Takagi–Sugeno–Kang fuzzy neural network (RGT2-TSKFNN) for the identification of nonlinear systems. In the proposed structure, the general type-2 fuzzy set (GT2FS) and a recurrent fuzzy neural network (RFNN) are combined to obviate the data uncertainties. The fuzzy firing strengths in the developed structure are returned to the network input as internal variables. In the proposed structure, GT2FS is utilized to characterize the antecedent parts while the consequent parts are performed using TSK type. The issues of constructing a RGT2-TSKFNN involve type reduction, structure learning as well as parameter learning. An efficient strategy is developed by utilizing alpha-cuts to decompose a GT2FS into several interval type-2 fuzzy sets (IT2FSs). In order to solve the computation time of the type-reduction issue, a direct defuzzification method is used instead of iterative nature of Karnik–Mendel​ (KM) algorithm. Type-2 fuzzy clustering and Lyapunov criteria are utilized for online structure learning as well as the antecedent and consequent parameters, respectively for reducing the number of rules and guaranteeing the stability of the proposed RGT2-TSKFNN. The reported comparative analysis of the simulation results is utilized to estimate the performance of the proposed RGT2-TSKFNN with respect to other popular type-2 FNNs (T2FNNs) methodologies. • We propose a recurrent general type-2 TSK fuzzy neural network. • The structure of the proposed network is learned online. • The Lyapunov stability criterion is used to update the proposed network parameters. [ABSTRACT FROM AUTHOR]

This paper proposes a spatial–temporal recurrent neural network architecture for deep Q -networks that can be used to steer an autonomous ship. The network design makes it possible to handle an arbitrary number of surrounding target ships while offering robustness to partial observability. Furthermore, a state-of-the-art collision risk metric is proposed to enable an easier assessment of different situations by the agent. The COLREG rules of maritime traffic are explicitly considered in the design of the reward function. The final policy is validated on a custom set of newly created single-ship encounters called 'Around the Clock' problems and the commonly used Imazu (1987) problems, which include 18 multi-ship scenarios. Performance comparisons with artificial potential field and velocity obstacle methods demonstrate the potential of the proposed approach for maritime path planning. Furthermore, the new architecture exhibits robustness when it is deployed in multi-agent scenarios and it is compatible with other deep reinforcement learning algorithms, including actor-critic frameworks. • Proposal of a spatial–temporal recurrent neural network architecture for the DQN. • Combination with a new collision risk metric to steer an autonomous ship. • Explicit consideration of COLREG rules for maritime traffic. • Validation on "Around the Clock", Imazu, and multi-agent scenarios. [ABSTRACT FROM AUTHOR]

In this paper, we present two-timescale neurodynamic optimization approaches to distributed minimax optimization. We propose four multilayer recurrent neural networks for solving four different types of generally nonlinear convex–concave minimax problems subject to linear equality and nonlinear inequality constraints. We derive sufficient conditions to guarantee the stability and optimality of the neural networks. We demonstrate the viability and efficiency of the proposed neural networks in two specific paradigms for Nash-equilibrium seeking in a zero-sum game and distributed constrained nonlinear optimization. [ABSTRACT FROM AUTHOR]

Although graph representation learning has been studied extensively in static graph settings, dynamic graphs are less investigated in this context. This paper proposes a novel integrated variational framework called DYnamic mixture Variational Graph Recurrent Neural Networks (DyVGRNN), which consists of extra latent random variables in structural and temporal modelling. Our proposed framework comprises an integration of Variational Graph Auto-Encoder (VGAE) and Graph Recurrent Neural Network (GRNN) by exploiting a novel attention mechanism. The Gaussian Mixture Model (GMM) and the VGAE framework are combined in DyVGRNN to model the multimodal nature of data, which enhances performance. To consider the significance of time steps, our proposed method incorporates an attention-based module. The experimental results demonstrate that our method greatly outperforms state-of-the-art dynamic graph representation learning methods in terms of link prediction and clustering. 2 2 The source code is available at https://github.com/GhazalehNiknam/DyVGRNN. [ABSTRACT FROM AUTHOR]