Search

Accurate prediction of structural response under earthquake is of great significance for structural damage and performance evaluation. In order to improve the efficiency of structure time history response prediction, this paper proposes a novel SeisFormer model based on the self-attention mechanism and deep learning technology. Through autoregressive prediction, the SeisFormer can achieve real-time prediction of the response time histories of a large number of nodes in the structure under seismic action and can effectively solve the problem of data scarcity. Four case studies are performed to verify the accuracy and efficiency of the proposed methodology, including validation on datasets obtained from elastoplastic seismic analysis of a single-story structure, a three-story structure, and an eleven-story structure, and measured data of a shaking table test model. In addition, this paper further studies the prediction accuracy of the SeisFormer through ablation experiments and comparative experiments. The experimental results show that the SeisFormer can accurately predict the acceleration, velocity, and displacement time histories of numerous nodes in the structure. The prediction accuracy outperforms the LSTM model, and the prediction speed is 193–109,824 times faster than finite element method. Furthermore, with data augmentation through autoregressive prediction, the SeisFormer model can achieve efficient and accurate predictions when training data is exceptionally scarce, enabling engineering applications. • A self-attention mechanism-based method for real-time refined structural response prediction. • An autoregressive data utilizing strategy. • A new deep learning model called SeisFormer. • The SeisFormer outperforms LSTM in structural response prediction under seismic action. • The SeisFormer predicts structural response 193 to 109,824 times faster than the traditional finite element method. [ABSTRACT FROM AUTHOR]

Link prediction is a fundamental problem in complex network analysis, aimed at predicting missing or forthcoming connections. Recent research investigates the potential of Nonnegative Matrix Factorization (NMF) models in reconstructing sparse networks and using deep NMF to uncover the hierarchical structure. Deep models have demonstrated remarkable performance in various domains, but they are susceptible to overfitting, especially on the limited training data. This paper proposes a novel Link Prediction using Adversarial Deep NMF (LPADNMF) to enhance the generalization of network reconstruction in sparse graphs. The main contribution is the introduction of an adversarial training that incorporates a bounded attack on the input, leveraging the L 2 , 1 norm to generate diverse perturbations. This adversarial training aims to improve the model's robustness and prevent overfitting, particularly in scenarios with limited training data. Additionally, the proposed method incorporates first and second-order affinities as input to capture higher-order dependencies and encourage the extraction of informative features from the network structure. To further mitigate overfitting, a smooth L 2 regularization is applied to the model parameters. To optimize the proposed model effectively, we utilize a majorization-minimization algorithm that efficiently updates the perturbation and latent factors in an iterative manner. Our findings demonstrate that the proposed model not only has the ability to uncover and learn complex structures but also possesses generalization capabilities. This method demonstrates superior performance compared to state-of-the-art methods across eight networks. In comparison to the best-performing approach, LPADNMF exhibited improvements of 2.36% in AUC, 4.67% in Precision, 2.21% in Recall, and 3.73% in F-Measure. [ABSTRACT FROM AUTHOR]

Executing various production tasks is critical to the safe operation and efficient production of industrial processes. As one of them, the detection task of key quality variables directly affects the operation optimization and decision-making of industrial processes, but it is severely limited by the harsh environment and detection instruments. Therefore, the real-time prediction task of key quality variables becomes the basis for optimal control of industrial processes. To address this issue, this paper proposes a task-oriented deep learning framework based on a target-related transformer (TR-Former) network for industrial quality prediction tasks. Specifically, a new target-related self-attention (TR-SA) mechanism is developed to guide feature learning by adding attention scores between task-related target variables and other variables. As a result, the learned features in this instance will be guaranteed to be relevant to the target variable and useful for the quality prediction task. Moreover, the long-range dynamics of industrial process data can also be captured, which can further improve the prediction performance of the model. Finally, extensive experiments were conducted on two industrial processes to validate the superiority of the proposed method in terms of quality prediction tasks. The experimental results demonstrate that the proposed TR-Former method exhibits an improvement ranging from 3% to 13% in the mean absolute error indicator compared to the traditional transformer and other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Accurate exchange rate forecasting is important for the better realization of international economic transactions and international currency investments. However, it is challenging to forecast exchange rates due to their high volatility, nonlinearity, and noisy characteristics. This paper creatively proposes a novel interpretable deep learning multi-dimensional integration framework for foreign exchange rate forecasting based on shallow and deep feature selection and snapshot ensemble technology. The main factors affecting exchange rates are considered at a deep and shallow level, and the feature selection project is implemented. A cluster-based multidimensional learning paradigm is proposed. The framework clusters data and constructs predictive sub-models and then embeds snapshot ensemble techniques into models to improve the robustness of single models. Finally, the results are nonlinearly integrated by a deep learning model. In addition, three datasets and twelve comparative models were used to demonstrate the performance of the models. The empirical results show that the proposed model has the smallest mean absolute percentage error (0.407859%, 0.47134%, and 0.470167%, respectively). [ABSTRACT FROM AUTHOR]

Ship Time Headway (STH) is used in maritime navigation to describe the time interval between the arrivals of two consecutive ships in the same water area. This measurement may offer a straightforward way to gauge the frequency of ship traffic and the likelihood of congestion in a particular area. STH is an important factor in understanding and managing the dynamics of ship movements in busy waterways. This paper introduces a hybrid deep learning method for predicting STH in time domain. The method integrates the Seasonal-Trend Decomposition using Loess (STL), Multi-head Self-Attention (MSA) mechanism into Long Short-Term Memory (LSTM) neural network. The STH dataset was extracted from the Automatic Identification System (AIS) through ship trajectory spatial motion, and the seasonal, trend and residual components of the decomposition were then determined from the STH dataset using the STL algorithms. MSA-LSTM is adopted to comprehensively capture the evolving patterns of STH from the sequence. Comparison studies with existing methods demonstrate the accuracy and robustness of the predictions provided by this method, indicating that the proposed method outperforms other models in terms of prediction performance and learning capabilities. By predicting STH, the method offers potential to assist maritime traffic managers and navigators in assessing ship flow, thereby enabling them to make informed decisions on navigation safety and efficiency. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]

Operational forecasting often requires predicting collections of related, multivariate time series data that are high-dimensional in nature. This can be tackled by fitting a single function to all series (global approach) or assuming each series as a separate prediction problem and fitting one function to each (local approach) – the global–local trade-off. Deep learning models inspired by different data generation processes aim to combine the benefits of global and local approaches. Specifically, these frequently propose feeding the statistical expressiveness of classical local models into more complex global networks. Following recent trends in neural networks, the theoretical foundations of these hybrid models can also explain the surge of Transformer-based time series forecasting applications, which showcase potential benefits for the global and local equilibrium. Dynamic reinforcement learning (RL) models have also been explored to optimize the balance of global and local signals in general prediction problems, frequently through Q-learning algorithms. RL models can be proposed to dynamically adjust the influence of global versus local information to improve predictive performance at both scales. This paper conducted a concise literature review focused on these two research streams to optimize the balance between globality and locality in forecasting collections of time series. It focuses on their evolution across time and hints at opportunities to close some of the research gaps by intersecting both propositions. We followed the Preferred Reporting Items for Systematic Reviews and Meta-analyzes (PRISMA) guidelines and achieved a selection of 143 publications since 2000. The main findings reveal that global models have achieved strong expressiveness in capturing the most complex structural patterns while still enabling probabilistic outcomes to be delivered through uncertainty estimates. On the other hand, RL based methods depict great benefits in mitigating the risks of generalization by imprinting contextual diversity when predicting each step ahead for each series. Within those, the adoption of other computational learning or evolutionary-based methods (e.g. Genetic Programming) to improve the parametrization of the learning policies is also highlighted as an area of future work yet to be uncovered. This review advances knowledge at the intersection of two distinct yet potentially complementary research areas, identifying opportunities to combine different methodological approaches in addressing the global–local trade-off in forecasting collections of time series. This is achieved by surfacing shared limitations in current research and presenting avenues for integrating distinct methodologies, namely by further developing the theoretical underpinnings of reinforcement learning techniques. With this work we seek to enhance understanding of the relevant research landscape and help inform future solutions by establishing a foundation for collaborative work. [Display omitted] • Review of global and local time series forecasting approaches including Deep Learning (DL) models combining global and local signals and Dynamic Reinforcement Learning (RL) methods optimizing this trade-off • Identification of opportunities to combine distinct forecasting methodologies like global DL models and RL based local modeling to address the global–local trade-off. • Search for ways to integrate RL techniques for time series forecasting by further developing their theoretical underpinnings based on limitations found in current research. • Main findings show global models achieve strong expressiveness capturing complex patterns while enabling probabilistic outcomes, and RL methods mitigate risks when predicting each step for each series [ABSTRACT FROM AUTHOR]

Executing various production tasks is critical to the safe operation and efficient production of industrial processes. As one of them, the detection task of key quality variables directly affects the operation optimization and decision-making of industrial processes, but it is severely limited by the harsh environment and detection instruments. Therefore, the real-time prediction task of key quality variables becomes the basis for optimal control of industrial processes. To address this issue, this paper proposes a task-oriented deep learning framework based on a target-related transformer (TR-Former) network for industrial quality prediction tasks. Specifically, a new target-related self-attention (TR-SA) mechanism is developed to guide feature learning by adding attention scores between task-related target variables and other variables. As a result, the learned features in this instance will be guaranteed to be relevant to the target variable and useful for the quality prediction task. Moreover, the long-range dynamics of industrial process data can also be captured, which can further improve the prediction performance of the model. Finally, extensive experiments were conducted on two industrial processes to validate the superiority of the proposed method in terms of quality prediction tasks. The experimental results demonstrate that the proposed TR-Former method exhibits an improvement ranging from 3% to 13% in the mean absolute error indicator compared to the traditional transformer and other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Ship Time Headway (STH) is used in maritime navigation to describe the time interval between the arrivals of two consecutive ships in the same water area. This measurement may offer a straightforward way to gauge the frequency of ship traffic and the likelihood of congestion in a particular area. STH is an important factor in understanding and managing the dynamics of ship movements in busy waterways. This paper introduces a hybrid deep learning method for predicting STH in time domain. The method integrates the Seasonal-Trend Decomposition using Loess (STL), Multi-head Self-Attention (MSA) mechanism into Long Short-Term Memory (LSTM) neural network. The STH dataset was extracted from the Automatic Identification System (AIS) through ship trajectory spatial motion, and the seasonal, trend and residual components of the decomposition were then determined from the STH dataset using the STL algorithms. MSA-LSTM is adopted to comprehensively capture the evolving patterns of STH from the sequence. Comparison studies with existing methods demonstrate the accuracy and robustness of the predictions provided by this method, indicating that the proposed method outperforms other models in terms of prediction performance and learning capabilities. By predicting STH, the method offers potential to assist maritime traffic managers and navigators in assessing ship flow, thereby enabling them to make informed decisions on navigation safety and efficiency. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]

Crude oil prices are susceptible to a variety of elements, including military, political, financial, and diplomatic forces, resulting in price fluctuations that are random, abrupt, and chaotic. Based on the chaotic nature of crude oil price series, this paper proposes a model for predicting crude oil prices that incorporates variational modal decomposition algorithm (VMD), phase space reconstruction technique (PSR), convolutional neural network (CNN), and bidirectional long and short-term memory network (BiLSTM). Specifically, the noises in the original data are removed using VMD. Next, the crude oil price is reconstructed using PSR. Then the reconstructed and denoised phase space is fed into the CNN-BiLSTM model for multi-step predictions. The empirical results show that the proposed model achieves the lowest MAPEs as 0.662%, 2.283%, 3.829%, and 4.44% and the lowest MSEs as 0.335, 3.942, 10.047 and 13.681. The Diebold Mariano (DM) test also demonstrates the proposed model's superiority in the domain of crude oil price forecasting. [ABSTRACT FROM AUTHOR]

Operational forecasting often requires predicting collections of related, multivariate time series data that are high-dimensional in nature. This can be tackled by fitting a single function to all series (global approach) or assuming each series as a separate prediction problem and fitting one function to each (local approach) – the global–local trade-off. Deep learning models inspired by different data generation processes aim to combine the benefits of global and local approaches. Specifically, these frequently propose feeding the statistical expressiveness of classical local models into more complex global networks. Following recent trends in neural networks, the theoretical foundations of these hybrid models can also explain the surge of Transformer-based time series forecasting applications, which showcase potential benefits for the global and local equilibrium. Dynamic reinforcement learning (RL) models have also been explored to optimize the balance of global and local signals in general prediction problems, frequently through Q-learning algorithms. RL models can be proposed to dynamically adjust the influence of global versus local information to improve predictive performance at both scales. This paper conducted a concise literature review focused on these two research streams to optimize the balance between globality and locality in forecasting collections of time series. It focuses on their evolution across time and hints at opportunities to close some of the research gaps by intersecting both propositions. We followed the Preferred Reporting Items for Systematic Reviews and Meta-analyzes (PRISMA) guidelines and achieved a selection of 143 publications since 2000. The main findings reveal that global models have achieved strong expressiveness in capturing the most complex structural patterns while still enabling probabilistic outcomes to be delivered through uncertainty estimates. On the other hand, RL based methods depict great benefits in mitigating the risks of generalization by imprinting contextual diversity when predicting each step ahead for each series. Within those, the adoption of other computational learning or evolutionary-based methods (e.g. Genetic Programming) to improve the parametrization of the learning policies is also highlighted as an area of future work yet to be uncovered. This review advances knowledge at the intersection of two distinct yet potentially complementary research areas, identifying opportunities to combine different methodological approaches in addressing the global–local trade-off in forecasting collections of time series. This is achieved by surfacing shared limitations in current research and presenting avenues for integrating distinct methodologies, namely by further developing the theoretical underpinnings of reinforcement learning techniques. With this work we seek to enhance understanding of the relevant research landscape and help inform future solutions by establishing a foundation for collaborative work. [Display omitted] • Review of global and local time series forecasting approaches including Deep Learning (DL) models combining global and local signals and Dynamic Reinforcement Learning (RL) methods optimizing this trade-off • Identification of opportunities to combine distinct forecasting methodologies like global DL models and RL based local modeling to address the global–local trade-off. • Search for ways to integrate RL techniques for time series forecasting by further developing their theoretical underpinnings based on limitations found in current research. • Main findings show global models achieve strong expressiveness capturing complex patterns while enabling probabilistic outcomes, and RL methods mitigate risks when predicting each step for each series [ABSTRACT FROM AUTHOR]

Foreign exchange rate affects many countries' economic status and development. Therefore, it is essential to find the factors affecting the exchange rate price and make reasonable predictions. This paper proposes the novel LASSO-BiLSTM-based ensemble learning method by integrating least absolute shrinkage and selection operator (LASSO) and bidirectional long short-term memory (LSTM) to predict the USD/CNY exchange rate. First, 29 variables are selected to reflect economic activities on market and macroeconomic levels. Then, LASSO-BiLSTM-based ensemble learning approach is adopted with two steps: 1) LASSO is used to select six highly correlated variables with the exchange rate to reduce noises. 2) BiLSTM is employed to forecast the exchange rate with the six chosen variables. Last, to test the effectiveness of BiLSTM, comparisons with four deep learning algorithms, which are extreme learning machine (ELM), kernel extreme learning machine (KELM), long short-term memory (LSTM), and support vector regression (SVR), are conducted. The result shows that LASSO-BiLSTM outperforms the other models in 1-step forecast (MAE: 0.051, RMSE: 0.072, MDA: 0.777). The same conclusion applies to 3-steps and 6-steps forecasts. Overall, the proposed LASSO-BiLSTM-based ensemble learning method demonstrates high potential in time series forecasting. [ABSTRACT FROM AUTHOR]

Astronomical seeing is the primary indicator of the observation quality of large-scale optical telescopes. Forecasting astronomical seeing is an essential and challenging task in the operation of large optical telescopes, as it can help observers more efficiently arrange observation tasks as well as providing hints for mechanism optimization of optical telescopes. In this paper, we propose an efficient data-driven astronomical seeing forecasting method by leveraging the environment monitoring data from multiple sensors deployed with optical telescopes. The basic idea is to use a large number of random yet fixed convolution kernels to extract features from the original data, and then train a simple tree-based predictor based on these features. In particular, the raw monitoring data is transformed by differential operations, enabling a more explicit representation of spatial and temporal features; then an elaborate convolution mechanism, including the convolution kernel and the pooling operator, is designed to extract a variety set of features of the input time series, which serve as the foundation for constructing the time series forecasting model. In contrast to existing deep learning models, there is no need to calculate and back-propagate the gradients during the training procedure, making the proposed method much more efficient. We collect real-world monitoring data from Large Sky Area Multi-Object Fiber Spectroscopy Telescope (LAMOST), one of the world's largest optical reflecting telescopes, and conduct extensive empirical evaluations. The experimental results demonstrate that the proposed method can achieve state-of-the-art prediction accuracy, i.e., 0.1125 in MAE, 0.0252 in MSE, and 0.0322 in MAPE, outperforming most deep-learning-based forecasting models. Moreover, it surpasses both machine learning and deep learning approaches in terms of training efficiency with a significant margin. [Display omitted] • The task of forecasting astronomical seeing for optical telescopes is investigated. • A variety of real-world monitoring data of LAMOST is collected. • A new multivariate time series forecasting method — SeeROCKET is proposed. • SeeROCKET achieves very promising performance both in accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Perceiving the future trend of Vessel Traffic Flow (VTF) in advance has great application values in the maritime industry. However, using such big data from the Automatic Identification System (AIS) for accurate VTF prediction remains challenging. Deep training networks can learn valuable features from extensive historical data. This paper proposes a new learning-based prediction network, improved Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) with similarity grouping, including three views. To effectively enable the training network to capture the temporal and periodic (i.e. a spatial attribute) change characteristics of VTF, the CNN and LSTM are employed to compose spatial and temporal views, respectively. Hence, the original one-dimensional data is transformed into a matrix (hour of the day ✕ day) to adapt the input of the proposed methodology. In practical applications, VTF of multiple adjacent target regions need to be predicted simultaneously, and the changes of VTF in different areas may influence each other. To explore their hidden relationships, the similarity grouping view aims to find the target area that exhibits the most similarity with the VTF change trend of the current research area. Furthermore, similar information is combined with the features generated from the other two views to obtain the prediction results. In summary, the new advantage lies in mining the spatiotemporal attributes of data and fusing the similarity information of adjacent regions. Comparative experiments with eleven other methods on realistic VTF datasets show that the proposed method demonstrates superior prediction accuracy and stability performance. • Multi-perspective systematic analysis of ship trajectory prediction research. • Comparative analysis of eleven methods to reveal their advantages and disadvantages. • Applicability of the eleven extracted methods under difficult circumstances. • New solutions to STP for future hybrid traffic of manned and unmanned ships. [ABSTRACT FROM AUTHOR]

This paper proposes a spatiotemporal graph neural network capable of effective representation learning of the spatiotemporal interrelationships and interdependencies of in-situ observation data from multiple locations for multivariate multi-step ahead time-series forecasting. The propose model is largely composed of graph learning, spatial encoder, and temporal decoder, and ablation studies on variants of the three modules and comparative experiments with state-of-the-art deep neural networks for sequence modeling were also performed extensively. The proposed model showed improved predictability than conventional numerical model-based approaches or state-of-the-art models by applying consecutive multi-step ahead time-series prediction of sea surface temperature at multiple locations along the coast. For more rigorous performance evaluation, not only the overall performance of the test data, but also the performance of extreme cases included in the test data based on historical records were separately assessed. The prediction rationales were also presented through quantified relative contributions between neighbor locations using the trained adjacency matrix obtained through graph learning. The results showed that it is well consistent with the ocean physics and geographical domain knowledge, demonstrating the feasibility and reliability of the proposed method. Therefore, the proposed method shows sufficient potential to be used as a scientific tool for decision-making in extreme events such as marine heat waves or for operational ocean forecasting. [ABSTRACT FROM AUTHOR]

Although research on time series prediction based on deep learning is being actively carried out in various industries, deep learning technology still has a high entry barrier for researchers who have not majored in computer science. This paper presents a tutorial on time series prediction using a deep learning-based model. The entire process of time series data prediction is presented—from data collection to evaluation of prediction results. The details of each step are shown through a case example of predicting peak electricity demand and system marginal price of Jeju Island in Korea using the 1D-CNN and BiLSTM model. In Jeju Island, the proportion of renewable energy in the total power generation capacity is increased to 67% in 2021, requiring more accurate electricity demand forecasts. Therefore, using 808 days of training data from February 2018, electricity demand and SMP for the next 21 days were predicted. To make it easier for readers to follow, the example uses only open public data, and the entire Python source code is shared via a GitHub repository. The prediction error calculated by WRMSSE showed 0.42 in electricity demand and 0.63 in SMP max. A WRMSSE value less than one means that the forecast was relatively good, that is, better than naïve forecasting. This tutorial is not limited to the energy industry but can be utilized for any application requiring time-series data prediction. This article is expected to be of great help to researchers who need to understand the process of time series prediction using deep learning and use it for application in their industry. [ABSTRACT FROM AUTHOR]

The prediction of the next activity in a business process can be very useful in revealing inefficiencies and take decisions to avoid undesired activities. In this direction, further advantages can gather from the knowledge of the activities within their attributes that influence the occurrence of the predicted activity. This paper faces this issue by introducing a Data-aware Explainable Next Activity Prediction approach called DENAP based on the adoption of Long Short-Term Memory (LSTM) neural networks and Layer-Wise Relevance Propagation (LRP) method for the detection of next activity prediction in a business process and for the evaluation of the past activities and data that influence the prediction. The DENAP approach is validated on a set of synthetic and real logs. The obtained results show the good capability of DENAP to predict the next activity and indicate the more relevant activities/data with respect to the prediction. [ABSTRACT FROM AUTHOR]

Process mining focuses on the analysis of event log data to build various process analytical capabilities. Predictive process analytics has emerged as one of such key capabilities and it uses machine learning techniques to construct process prediction models. In recent years, deep neural networks have gained increasing interest in process prediction since they can handle multi-dimensional sequential inputs with minimal information loss. However, they are considered black-box models and existing studies in explaining deep neural network-based process predictions rely on only event-level features for explanation. In this paper, we propose a new approach for generating explanations for process outcome predictions at multiple levels. The approach is underpinned by three different prediction models: a transparent model for generating global explanations based on case-level features, an attention-based deep neural network for generating local explanations based on event-level features, and a novel eXplainable Dual-learning Deep network (XD 2 -net) for generating local explanations based on case-level features. Using three publicly available datasets, we have tested the applicability of the approach and further examined the multi-level explanations generated by the approach through an elaborate case study. Unlike others, the design of our approach promotes the idea of leveraging the complementary capabilities of different models and utilizing their strengths, rather than focusing on model performance competition. This will contribute towards generating more comprehensive explanations that meet the needs of different end users and purposes in the future. [ABSTRACT FROM AUTHOR]

As the complexity of power systems increases, traditional model-driven methods for online frequency stability prediction (FSP) encounter constraints in both accuracy and efficiency. To enhance the accuracy and efficiency of FSP, an data-driven method using CoAtNet and SHAP values is proposed. By leveraging the combination of convolution and attention mechanisms, CoAtNet addresses the limitation of traditional deep learning approaches that may not be able to extract data features comprehensively. Moreover, selecting all features as input into a deep-learning model may cause a substantial computation burden. It is thus impractical for CoAtNet to perform FSP of large-scale power systems. For this problem, this paper develops a SHAP values-based feature selection method to select the effective features as input. This process greatly reduces the numerical complexity, maintaining a high prediction performance. Additionally, the marginally stable situation of the system frequency is ignored by most researchers. A frequency security index to identify marginally stable situations is thus employed to generate the data labels, which are classed as "absolute security", "relative security", and "insecurity". Finally, verified by the comparison simulation, the proposed model outperforms other models with accuracies of 98.80% on the modified IEEE 39-bus system and 99.04% on the modified ACTIVSg500 system. [ABSTRACT FROM AUTHOR]

The increase of road traffic in large cities during the last years has produced that long and short-term traffic flow forecasting is a critical need for the authorities. The availability of good traffic flow prediction methods is a must to make informed decisions concerning (punctual) traffic congestions. Previous work has shown that the accuracy of these methods decreases if we consider urban traffic and long-term predictions. In this paper we present a hybrid model, combining a Convolutional Neural Network and a Bidirectional Long–Short-Term Memory network, and apply it to long-term traffic flow prediction in urban routes. This model combines the capability of CNN to extract hidden valuable features from the input model and the capability of BiLSTM to understand the temporal context. In order to assess the usefulness of our model, we considered four streets of the city of Madrid with different characteristics and compared the results of our proposed model with the ones obtained by eight widely used baseline models. The results show that our hybrid model outperforms the baseline models with respect to three metrics commonly used in regression: mean absolute error, root mean squared error and accuracy. [ABSTRACT FROM AUTHOR]

Accurate multi-node load forecasting is the key to the safe, reliable, and economical operation of the power system. However, the dynamic nature of load and the coupling nature of networks are difficult to extract, making consistent and accurate forecasting of node load rather difficult. In this regard, this paper proposes a soft sharing multi-task deep learning method for multi-node load forecasting in the power system. It has the following aspects: (1) Considering the coupling characteristics of the node network, a multi-modal feature module, based on the inception strategy and gated temporal convolutional network (GTCN), is firstly designed to explore the coupling features implied in the node load data. (2) A novel multi-objective neural network model is proposed to achieve simultaneous prediction of multi-node load by integrating the multi-modal feature module and gated recurrent unit (GRU). For sharing the learning information of sub-networks, this paper uses the soft sharing mechanism to capture load features, which can better optimize the prediction task for each node load simultaneously. Load data from the New Zealand distribution network and AEMO are used to compare the proposed model's performance in various scenarios using regression metrics such as mean absolute percentage error (MAPE), Weighted Mean Accuracy (WMA), root mean squared logarithmic error (RMSLE), and Diebold–Mariano (DM). The simulation results show that the proposed method can explore the spatial–temporal coupling characteristics in multi-node load data. Compared with existing state-of-the-art multi-node load prediction methods, our proposed method's MAPE decrease 17.04% and 3.92% in Non-aggregation and Aggregation situations. • A novel deep multi-task learning method is proposed for the multi-node forecasting problem. • An IGTCN module is designed to extract the coupling features from different node loads. • The multi-task deep neural network exhibits the accuracy of multi-node load forecasting. [ABSTRACT FROM AUTHOR]

While constructing predictive model for industrial process quality prediction, the selection of an appropriate input variable set is crucial to the online prediction performance. In this paper, the issue is solved through a deep causality analysis method, where the deep learning feature extracting unit, i.e., the Gated Recurrent Unit (GRU), is combined with the Granger Causality (GC) to extract the causal relationships between process variables. In this way, the nonlinear and dynamic features contained in the process data can be effectively learned in the GC analyzing framework, resulting in a more reliable causal relationship structure. Furthermore, through applying the attention mechanism to the input layer of the GRU cells, the causal relationship between process variables and the quality variable can be extracted, even the long time-lags exist. Based on the direct and indirect causal structure, the quality-related causal input variables are selected, and the predictive model is established through a deep dynamic regression network with GRU cells. A numerical example and a real chemical process case are provided to verify the effectiveness of the proposed causal variable selection method, where the prediction performance is significantly improved comparing with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The reliable control of wave energy devices highly relies on the forecasts of wave heights. However, the dynamic characteristics and significant fluctuation of waves' historical data pose challenges to precise predictions. Neural networks offer a promising solution to forecast the wave heights by extracting meaningful features from historical observations. This paper proposes a novel hybrid random vector functional link network with the ensemble and deep learning benefits. Hierarchical stacks of hidden layers are constructed to enforce the deep representations of the time series. Individual output layers follow all enhancement layers to adopt ensemble learning. A neuron pruning strategy is proposed to remove the noisy information from the random features and boost the network's performance. Besides, the proposed network is further utilized to forecast the additive and multiplicative residuals from the ARIMA method. Finally, the ensemble of additive-ARIMA-edRVFL, multiplicative-ARIMA-edRVFL, and edRVFL achieves the best average rankings around two for three forecasting horizons. The proposed ensemble achieves an average ranking of 1.33 on four-hours ahead of forecasting in terms of root mean square error and mean absolute scaled error. Extensive experiments are conducted on twelve time series of the significant wave height. The comparative results demonstrate the superiority of the proposed model over other state-of-the-art methods. The source codes are available on https://github.com/P-N-Suganthan/CODES. [ABSTRACT FROM AUTHOR]

The importance of considering related stocks data for the prediction of stock price movement has been shown in many studies; however, advanced graphical techniques for modeling, embedding and analyzing the behavior of inter-related stocks have not been widely exploited for the prediction of stocks price movements yet. The main challenges in this domain are to find a way for modeling the existing relations among an arbitrary set of stocks and to exploit such a model for improving the prediction performance for those stocks. The most of existing methods in this domain rely on basic graph-analysis techniques, with limited prediction power, and suffer from a lack of generality and flexibility. In this paper, we introduce a novel framework, called GCNET that models the relations among an arbitrary set of stocks as a graph structure called influence network and uses a set of history-based prediction models to infer plausible initial labels for a subset of the stock nodes in the graph. Finally, GCNET uses the Graph Convolutional Network algorithm to analyze this partially labeled graph and predicts the next price direction of movement for each stock in the graph. GCNET is a general prediction framework that can be applied for the prediction of the price fluctuations of interacting stocks based on their historical data. Our experiments and evaluations on a set of stocks from the NASDAQ index demonstrate that GCNET improves the performance of the state-of-the-art algorithms in terms of Accuracy and Matthew's Correlation Coefficient by at least 1.5% and 2%, respectively. • GCNET is a novel graph-based framework for prediction of stock price fluctuations. • GCNET introduce a novel approach for modeling stocks relations as a network. • GCNET applies the novel PLD mechanism to predict reliable initial labels for nodes in a network of stocks. [ABSTRACT FROM AUTHOR]

Froth Flotation is one of the fundamental processes used in iron ore attainment. It involves various physical–chemical reactions, and its multivariate control is not trivial. Traditionally, plant operators get quality measurements from laboratory analysis of sampled material, which can take hours to be published. In this paper, we present alternatives that eliminate both the necessity to periodically sample material as well as the delay associated with laboratory analysis: a predictive machine learning model working as a soft sensor. Two different classes of models were developed, exploring both deep and shallow approaches of machine learning, with a deep neural network architecture constructed for the particular task at hand. All models yielded accurate predictions, with results favoring the deep neural network when a larger volume of data is available. [ABSTRACT FROM AUTHOR]

It will be beneficial to devise an effective approach for short-term macroeconomic forecasting. Existing traditional statistics-based macroeconomic forecasting mainly focuses on exploring feasible methods for improving the accuracy of long-term predictions. However, the performance of short-term predictions was far less impressive under the impact of unexpected incidents. Furthermore, some deep learning-based approaches can achieve fine-grained variable frequency forecasting and preliminarily demonstrate effective results, but the interpretability is still controversial. Therefore, how to consider both the performance and the interpretability has already become a universal concern and an urgent unsolved problem. In this paper, we identified the above issue and proposed an interpretable data-driven approach, named EcoForecast 1 1 The code and data are available at https://github.com/navfour/ecoforecast. , for short-term macroeconomic forecasting based on the N-BEATS (neural basis expansion analysis for interpretable time series forecasting) neural network. To the best of our knowledge, EcoForecast is the first interpretable purely data-driven unified normative scheme for macroeconomic forecasting that achieves variable forecast frequencies and prediction domains, surpassing traditional statistics-based and deep learning-based approaches in performance or interpretability. EcoForecast used a three-level hierarchical signal encoding, including the fully connected neural network (FCNN) level, the block level, and the stack level. The FCNN level implemented both forecast and backcast information extraction for the temporal prediction and parameter learning of context. Block levels were connected by residuals so that the block's backcast could be sequentially filtered on the input. The stack level was used to form the top-level system of EcoForecast, where each stack was constrained to specialize in different inductive functions. To some extent, EcoForecast can balance effectiveness, efficiency, generalizability and interpretability while conditions such as forecast frequency and time window change, even when unexpected incidents occur. Based on the actual macroeconomic data for China from 1992 to 2022, the data-driven EcoForecast demonstrated high stability in different sequence learning scenarios and the accompanying high-accuracy performance. This stability was reflected in smaller prediction error expectation and variance, tolerance of fewer input samples, and robustness across prediction domains. The experimental results indicated that EcoForecast improved the accuracy up to 3.94 times compared with the traditional BVAR. In the robustness test, EcoForecast required only a quarter of the data to achieve 2.51 times smaller forecast errors than the BVAR while also improving the accuracy of varied macroeconomic indicators such as the Purchasing Managers' Index (PMI) and national electricity generation (ELEC)forecasting by 2.38 and 1.45 times. EcoForecast had a high sensitivity to the emergence of economic inflection points and adapts quickly when the economic environment changes, thus demonstrating a performance that exceeds traditional solutions in GDP forecasting during epidemics and PMI forecasting during economic turmoil. Interfacing with traditional economics research, the interpretable EcoForecast can uncover the trends and cycles of economic change, from which the conclusions are validated with the actual economics practice in China. Our findings can provide a new possible research direction for short-term macroeconomic forecasting. [ABSTRACT FROM AUTHOR]