Search

This paper proposes a three-stage online deep learning model for time series based on the ensemble deep random vector functional link (edRVFL). The edRVFL stacks multiple randomized layers to enhance the single-layer RVFL's representation ability. Each hidden layer's representation is utilized for training an output layer, and the ensemble of all output layers forms the edRVFL's output. However, the original edRVFL is not designed for online learning, and the randomized nature of the features is harmful to extracting meaningful temporal features. In order to address the limitations and extend the edRVFL to an online learning mode, this paper proposes a dynamic edRVFL consisting of three online components, the online decomposition, the online training, and the online dynamic ensemble. First, an online decomposition is utilized as a feature engineering block for the edRVFL. Then, an online learning algorithm is designed to learn the edRVFL. Finally, an online dynamic ensemble method, which can measure the change in the distribution, is proposed for aggregating all layers' outputs. This paper evaluates and compares the proposed model with state-of-the-art methods on sixteen time series. [ABSTRACT FROM AUTHOR]

The lack of intuitive and active human–robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study's detailed analysis can improve the usability of the assistive/rehabilitation robots. [ABSTRACT FROM AUTHOR]

• We propose an automated layer-wise pipeline for randomized feed-forward neural networks. • We propose a Sample-level Attention Strategy to solve the capabilities problem. • Several ablation experiments are conducted to verify the effect of hyper-parameters. The randomized feed-forward neural network is a single hidden layer feed-forward neural network that enables efficient learning by optimizing only the output weights. The ensemble deep learning framework significantly improves the performance of randomized neural networks. However, the framework's capabilities are limited by traditional hyper-parameter selection approaches. Meanwhile, different random network architectures, such as the existence or lack of a direct link and the mapping of direct links, can also strongly affect the results. We present an automated learning pipeline for the ensemble deep randomized feed-forward neural network in this paper, which integrates hyper-parameter selection and randomized network architectural search via Bayesian optimization to ensure robust performance. Experiments on 46 UCI tabular datasets show that our strategy produces state-of-the-art performance on various tabular datasets among a range of randomized networks and feed-forward neural networks. We also conduct ablation studies to investigate the impact of various hyper-parameters and network architectures. [ABSTRACT FROM AUTHOR]

The volatile characteristics of the tanker market pose challenges to forecasting. In addition, the volatile characteristics of newbuilding and secondhand ship prices, time charter rates, and scrap values make developing a unified framework of forecasting difficult. Most researchers have developed forecasting models and evaluated their performance based on a specific market. Such narrow development imposes difficulty for practitioners to choose a suitable model. Due to the boom of machine learning, many researchers are trying to boost the forecasting accuracy of shipping markets using machine learning. However, there are many hyper-parameters of the complex machine learning models and a slight variation of the model may cause significant performance degradation. This paper utilizes a forecast combination mechanism to forecast many time series collected from the shipping market, including newbuilding and secondhand ship prices, scrap values, and time charter rates. The models inside the combination pool are just linear functions. Finally, we compare their performance with conventional machine learning models and naïve forecasts using three error metrics and statistical tests. The statistical tests show that the combination of linear models is superior. The findings of this study also indicate that complex models do not boost forecasting accuracy necessarily. [ABSTRACT FROM AUTHOR]

Hospitals can predetermine the admission rate and facilitate resource allocation based on valid emergency requests and bed capacity estimation. The excess unoccupied beds can be determined with the help of forecasting the number of discharged patients. Extracting predictive features and mining the temporal patterns from historical observations are crucial for accurate and reliable forecasts. Machine learning algorithms have demonstrated the ability to learn temporal knowledge and make predictions for unseen inputs. This paper utilizes several machine learning algorithms to forecast the inpatient discharges of Singapore hospitals and compare them with statistical methods. A novel ensemble deep learning algorithm based on random vector functional links is established to predict inpatient discharges. The ensemble deep learning framework is optimized in a greedy layer-wise fashion. Several forecasting metrics and statistical tests are utilized to demonstrate the proposed method’s superiority. The proposed algorithm statistically outperforms the benchmark with a ranking of 1.875. Finally, practical implications and future directions are discussed. [ABSTRACT FROM AUTHOR]

Financial time series forecasting is crucial in empowering investors to make well-informed decisions, manage risks effectively, and strategically plan their investment activities. However, the non-stationary and non-linear characteristics inherent in time series data pose significant challenges when accurately predicting future forecasts. This paper proposes a novel Recurrent ensemble deep Random Vector Functional Link (RedRVFL) network for financial time series forecasting. The proposed model leverages randomly initialized and fixed weights for the recurrent hidden layers, ensuring stability during training. Furthermore, incorporating stacked hidden layers enables deep representation learning, facilitating the extraction of complex patterns from the data. The proposed model generates the forecast by combining the outputs of each layer through an ensemble approach. A comparative analysis was conducted against several state-of-the-art models over financial time-series datasets, and the results demonstrated the superior performance of our proposed model in terms of forecasting accuracy and predictive capability. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]

Neural networks have been successfully employed in various domains such as classification, regression and clustering, etc. Generally, the back propagation (BP) based iterative approaches are used to train the neural networks, however, it results in the issues of local minima, sensitivity to learning rate and slow convergence. To overcome these issues, randomization based neural networks such as random vector functional link (RVFL) network have been proposed. RVFL model has several characteristics such as fast training speed, direct links, simple architecture, and universal approximation capability, that make it a viable randomized neural network. This article presents the first comprehensive review of the evolution of RVFL model, which can serve as the extensive summary for the beginners as well as practitioners. We discuss the shallow RVFLs, ensemble RVFLs, deep RVFLs and ensemble deep RVFL models. The variations, improvements and applications of RVFL models are discussed in detail. Moreover, we discuss the different hyperparameter optimization techniques followed in the literature to improve the generalization performance of the RVFL model. Finally, we present potential future research directions/opportunities that can inspire the researchers to improve the RVFL's architecture and learning algorithm further. • The first survey focusing solely on RVFL-based models. • RVFLs in shallow, ensemble, deep, and ensemble deep frameworks have been discussed. • Various applications of the RVFL have been discussed. • Hyper-parameter optimization and experimental setups for the RVFL are discussed. • We present potential future research directions for the RVFL model. [ABSTRACT FROM AUTHOR]

Accurate forecasting of the air quality index (AQI) plays a crucial role in taking precautions against upcoming air pollution risks. However, air quality may fluctuate greatly in a certain period. Existing forecasting approaches always face the problem of losing valuable information on air quality status, even in the interval models of recent research. To address this issue, this paper suggests a new AQI forecasting perspective and paradigm built upon ternary interval-valued time series (TITS), multivariate variational mode decomposition (MVMD), multivariate relevance vector machine (MVRVM), mixed coding particle swarm optimization (MCPSO), and meteorological factors, which is able to capture the trend and volatility changes of AQI concurrently. The proposed paradigm involves four procedures: TITS construction in terms of the daily minimum, daily mean, and daily maximum AQI, multi-scale decomposition via MVMD, individual forecasting by MCPSO-optimized MVRVM, and ensemble learning forecasting using a simple addition approach. Experiments based on datasets collected from four municipalities in China demonstrated that the presented paradigm can hit higher accuracy than other comparable models, and the application analysis also shows that it has application potential in the AQI online forecasting system. To conclude, the proposed paradigm provides a promising alternative for AQI time series forecasting. • A ternary interval decomposition ensemble learning paradigm for AQI time series forecasting. • Multivariate variational mode decomposition is applied to thoroughly analyze the TIAQITS. • Multivariate relevance vector machine is introduced to model the three bounds of TIAQITS. • Mixed coding particle swarm optimization is adopted to optimize the parameters of MVRVM. [ABSTRACT FROM AUTHOR]

Forecasts of the wave heights can assist in the data-driven control of wave energy systems. However, the dynamic properties and extreme fluctuations of the historical observations pose challenges to the construction of forecasting models. This paper proposes a novel dynamic ensemble deep Echo state networks (ESN) to learn the dynamic characteristics of the significant wave height. The dynamic ensemble ESN creates a profound representation of the input and trains an independent readout module for each reservoir. To begin, numerous reservoir layers are built in a hierarchical order, adopting a reservoir pruning approach to filter out the poorer representations. Finally, a dynamic ensemble block is used to integrate the forecasts of all readout layers. The suggested model has been tested on twelve available datasets and statistically outperforms state-of-the-art approaches. • A dynamic ensemble deep echo state network is proposed. • A reservoir pruning strategy is proposed to remove inferior features. • An extensive review of significant wave height forecasting is conducted. • Our model manifests better performance than the SOTA models. [ABSTRACT FROM AUTHOR]

Electric load forecasting is essential for the planning and maintenance of power systems. However, its un-stationary and non-linear properties impose significant difficulties in predicting future demand. This paper proposes a novel ensemble deep Random Vector Functional Link (edRVFL) network for electricity load forecasting. The weights of hidden layers are randomly initialized and fixed during the training process. The hidden layers are stacked to enforce deep representation learning. Then, the model generates the forecasts using the ensemble of the outputs of each layer. Moreover, we also propose to augment the random enhancement features by empirical wavelet transformation (EWT). The raw load data are decomposed by EWT in a walk-forward approach without introducing future data leakage problems in the decomposition process. Finally, all the sub-series generated by the EWT, including raw data, are fed into the edRVFL for forecasting purposes. The proposed model is evaluated on sixteen publicly available time series from the Australian Energy Market Operator of the year 2020. The simulation results demonstrate the proposed model's superior performance over eleven forecasting methods in two error metrics and statistical tests on electricity load forecasting tasks. • An ensemble deep randomized network for forecasting is proposed. • Empirical wavelet transformation is used to augment unsupervised features. • A layer-wise tuning algorithm is proposed to optimize the structures. • Two error metrics and statistical tests are utilized to compare the models. [ABSTRACT FROM AUTHOR]

Asset prices play a significant role in the financial survival and profitability of ship-owning firms. In a highly volatile shipping market, prices of newbuilding ships must be predicted to detect security shortfalls as well as opportunities for temporal arbitration (gaining on high–low pricing). Accordingly, this paper proposes an improved version of the intelligent model search engine (IMSE) by asynchronous time lag selection. The parsimonious IMSE algorithm comprises the essential components such as input and training data size selection by a grid search procedure. In the initial IMSE algorithm, time-lag (memory size) selection is designed such that a serial cluster of memory groups is assigned synchronously for all inputs. By relaxing of lag structures selection, the proposed algorithm estimates unique lead–lag relations for the input of the intended problem set. An extensive benchmark study with several baseline models and the persistence forecast (Naïve I) is performed to observe the out-of-sample accuracy of the proposed approach. The empirical results indicate that second-hand ship prices, scrap values, and orderbook (no. of orders) have predictive features and are selected by the search engine for two ship sizes. Different lag structures are estimated for each input with asynchronous time-lag selection improvement. • A novel intelligent model search engine algorithm is proposed for newbuilding ship prices. • The proposed algorithm ensures the parsimony of the lag structure. • The selected lags and explanatory variables are highly interpretable. [ABSTRACT FROM AUTHOR]

Time charter rates must be predicted accurately to assist sensible decisions in the global, highly volatile shipping market. Time charter rates are affected by multiple factors, such as second-hand ship prices, order book, Libor interest rate, etc. However, not all factors convey predictive features to anticipate the future of time charter rates. Therefore, extracting predictive features from multiple driving time series from the shipping market is crucial for forecasting purposes. Accordingly, this paper proposes a novel convolutional recurrent neural network for time charter rates forecasting under the multi-variate phenomenon. The proposed network first extracts features from the monthly time series using a novel convolutional filter, the annual dilated filter. The annual dilated convolutional filter can extract the predictive features effectively and impose a sparse input structure to prevent overfitting. Then, a recurrent neural network learns the temporal information from the convoluted features. An extensive comparison study with many baseline models, including the persistence (Naïve I), statistical models, and the state-of-art networks, is conducted on the time charter rates of six kinds of ships. The empirical results demonstrate the proposed model's superiority in forecasting the time charter rates. • A novel dilated convolution LSTM neural network is proposed to forecast time charter rates. • The proposed convolutional filters can extract temporal features of monthly charter rates efficiently. • The proposed model outperforms the baselines in terms of three error metrics. [ABSTRACT FROM AUTHOR]

The challenge of accurately forecasting a time series covers numerous disciplines, from economics to engineering. Among the thousands of machine learning models, random vector functional link (RVFL) is a robust and efficient model which has demonstrated its success in various challenging forecasting problems. RVFL is an efficient universal function appropriator that randomly generates the weights between the input and hidden layers. However, RVFL still lacks the strong ability to extract meaningful multi-scale features from input data because of the single-layer random mapping of enhancement nodes. Therefore, we propose to combine the empirical wavelet transformation (EWT) with RVFL to strengthen the multi-scale feature extraction ability. The EWT can decompose the original time series into several sub-series which carry the information of different frequencies. Besides, we propose a walk-forward decomposition mechanism to implement the EWT. By introducing such a walk-forward mechanism and the combination of EWT and RVFL, the hybrid model achieves high accuracy and averts the data leakage problem during forecasting. A detailed and comprehensive empirical study on twenty-six public time series validates the proposed model's superiority compared with ten popular baseline models from the literature. • A causal signal decomposition method is proposed to avert the data leakage problem. • The empirical wavelet transformation is applied to decompose the time series. • A novel hybrid RVFL network is proposed for time series forecasting. [ABSTRACT FROM AUTHOR]

Fuzzy cognitive maps have achieved significant success in time series modeling and forecasting. However, fuzzy cognitive maps still contain weakness to handle the nonstationarity and outliers. We propose a novel time series forecasting model based on fuzzy cognitive maps and empirical wavelet transformation in this paper. The empirical wavelet transformation is applied to decompose the original time series into different levels which capture information of different frequencies. Then, the high-order fuzzy cognitive map is trained to model the relationships among all the sub-series generated and original time series. To enhance the robustness of high-order fuzzy cognitive maps against outliers, a novel learning method based on support vector regression is designed. Finally, we divide the summation of each concept value of the high-order fuzzy cognitive map by two to obtain the numerical predictions. A comprehensive empirical study on eight public time series validates the superiority of proposed model compared with the popular baseline models from the literature. • A novel EWT-FCM is proposed to time series forecasting. • The EWT is developed to decompose the original time series. • A robust learning algorithm is designed for high-order FCM. • Proposed model is validated through an empirical study of various time series. [ABSTRACT FROM AUTHOR]

Deep learning and fuzzy models provide powerful and practical techniques for solving large-scale deep-learning tasks. The fusion technique on deep learning and fuzzy system are generally classified into ensemble and integrated modes and materializes in information fusion, model fusion, and feature fusion. In an ensemble-based fusion, the fuzzy model either acts as an activation function or is operated as a separate process aggregating/preprocessing the information. Some early attempts in the field have successfully fused deep neural networks and fuzzy modeling concepts in ensemble mode. However, no effective attempts were made to fuse fuzzy models as an integrated feature-level fusion learning with graph neural networks (GNNs). This is mainly due to two challenges related to this fusion: (1) the number of fuzzy rules grows exponentially with the number of features that causes computational inefficiency, and (2) the solution space created by this fusion of fuzzy rules becomes complex due to multiple regression relations between inputs and outputs. Additionally, a simple linear regression at the output space would not be sufficient to model deep learning tasks. Therefore, this paper addresses these challenges by proposing a feature-level fusion method to fuse deep learning and fuzzy modeling where the latter technique is for integrated feature learning, called fuzzy forest graph neural network (FuzzyGNN), which creates a fuzzy learning forest fusing the linear graph transformers for deep learning tasks. We conducted experiments on fourteen machine learning datasets to test and validate the efficiency of the proposed FuzzyGNN model. Compared to state-of-the-art methods, our algorithm achieves the best results on four out of five machine learning datasets. The source code will be available at https://github.com/lingping-fuzzy/ , https://github.com/vojha-code and https://github.com/P-N-Suganthan. • We present an integrated fusion method for large-scale dataset classification. • We propose an integrated fusion of fuzzy learning and graph neural networks. • We propose sampling methods to address fuzzy rules exponentially explosion problem. • We propose adopting polynomial functions to simulate fitting prediction. • We experimentally justify the performance of the proposed model. [ABSTRACT FROM AUTHOR]