Search

This paper proposes a three-stage online deep learning model for time series based on the ensemble deep random vector functional link (edRVFL). The edRVFL stacks multiple randomized layers to enhance the single-layer RVFL's representation ability. Each hidden layer's representation is utilized for training an output layer, and the ensemble of all output layers forms the edRVFL's output. However, the original edRVFL is not designed for online learning, and the randomized nature of the features is harmful to extracting meaningful temporal features. In order to address the limitations and extend the edRVFL to an online learning mode, this paper proposes a dynamic edRVFL consisting of three online components, the online decomposition, the online training, and the online dynamic ensemble. First, an online decomposition is utilized as a feature engineering block for the edRVFL. Then, an online learning algorithm is designed to learn the edRVFL. Finally, an online dynamic ensemble method, which can measure the change in the distribution, is proposed for aggregating all layers' outputs. This paper evaluates and compares the proposed model with state-of-the-art methods on sixteen time series. [ABSTRACT FROM AUTHOR]

• We propose an automated layer-wise pipeline for randomized feed-forward neural networks. • We propose a Sample-level Attention Strategy to solve the capabilities problem. • Several ablation experiments are conducted to verify the effect of hyper-parameters. The randomized feed-forward neural network is a single hidden layer feed-forward neural network that enables efficient learning by optimizing only the output weights. The ensemble deep learning framework significantly improves the performance of randomized neural networks. However, the framework's capabilities are limited by traditional hyper-parameter selection approaches. Meanwhile, different random network architectures, such as the existence or lack of a direct link and the mapping of direct links, can also strongly affect the results. We present an automated learning pipeline for the ensemble deep randomized feed-forward neural network in this paper, which integrates hyper-parameter selection and randomized network architectural search via Bayesian optimization to ensure robust performance. Experiments on 46 UCI tabular datasets show that our strategy produces state-of-the-art performance on various tabular datasets among a range of randomized networks and feed-forward neural networks. We also conduct ablation studies to investigate the impact of various hyper-parameters and network architectures. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]

Efficient forecasting of ship order books holds immense significance in the maritime industry, enabling companies to optimize their operations, allocate resources effectively, and make informed decisions. However, volatile characteristics within historical order books pose challenges in achieving reliable, intelligent, and precise forecasts. This paper presents a novel ensemble deep random vector functional link (edRVFL) algorithm to anticipate future ship order book dynamics. The edRVFL leverages deep feature extraction and ensemble learning to enhance forecasting performance. To further elevate its capabilities, we introduce a discontinuous and parsimonious embedding strategy, which deviates from the conventional dense collection of continuous time steps used in vanilla edRVFL. This parsimonious embedding approach limits the model's complexity and boosts its generalization ability. We extensively evaluate the proposed method using ship order book data, and comparative studies demonstrate its superiority over alternative approaches. Our proposed edRVFL offers a promising solution for accurate and efficient ship order book forecasting, making it a valuable asset in the maritime industry's decision-making processes. The source codes utilized in this research are openly available on GitHub at the following link: https://github.com/crkkkaa/Ship-order-book-forecasting-by-an-ensemble-deep-parsimonious-random-vector-functional-link-network-. • A novel ensemble deep RVFL to anticipate ship order book dynamics is proposed. • We propose a parsimonious embedding algorithm to optimize temporal patterns. • The direct links ablation study is conducted for comparison. [ABSTRACT FROM AUTHOR]

Randomized neural networks (RNNs) have shown outstanding performance in many different fields. The superiority of having fewer training parameters and closed-form solutions makes them popular in small datasets analysis. However, automatically decoding raw electroencephalogram (EEG) data using RNNs is still challenging in EEG-based passive brain–computer interface (pBCI) classification tasks. Models with the high-dimension input of EEG may suffer from overfitting and the intrinsic characteristics of non-stationary, high-level noises and subject variability could limit the generation of distinctive features in the hidden layers. To address these problems in EEG-based pBCI tasks, this work proposes a spectral-ensemble deep random vector functional link (SedRVFL) network that focuses on feature learning in the frequency domain. Specifically, an unsupervised feature-refining (FR) block is proposed to improve the low feature learning capability in RNNs. Moreover, a dynamic direct link (DDL) is performed to further complement the frequency information. The proposed model has been evaluated on a self-collected dataset as well as a public driving dataset. The cross-subject classification results obtained demonstrated its effectiveness. This work offers a new solution for EEG decoding, i.e. , using optimized RNNs for decoding complex raw EEG data and boosting the classification performance of EEG-based pBCI tasks. • A SedRVFL network is proposed for the decoding of high-dimension raw EEG data. • A FR block is presented to improve the feature learning capability of the edRVFL. • Non-linear features are employed in the direct link. • A dynamic direct link algorithm is proposed to reduce the redundant information. [ABSTRACT FROM AUTHOR]

The reliable control of wave energy devices highly relies on the forecasts of wave heights. However, the dynamic characteristics and significant fluctuation of waves' historical data pose challenges to precise predictions. Neural networks offer a promising solution to forecast the wave heights by extracting meaningful features from historical observations. This paper proposes a novel hybrid random vector functional link network with the ensemble and deep learning benefits. Hierarchical stacks of hidden layers are constructed to enforce the deep representations of the time series. Individual output layers follow all enhancement layers to adopt ensemble learning. A neuron pruning strategy is proposed to remove the noisy information from the random features and boost the network's performance. Besides, the proposed network is further utilized to forecast the additive and multiplicative residuals from the ARIMA method. Finally, the ensemble of additive-ARIMA-edRVFL, multiplicative-ARIMA-edRVFL, and edRVFL achieves the best average rankings around two for three forecasting horizons. The proposed ensemble achieves an average ranking of 1.33 on four-hours ahead of forecasting in terms of root mean square error and mean absolute scaled error. Extensive experiments are conducted on twelve time series of the significant wave height. The comparative results demonstrate the superiority of the proposed model over other state-of-the-art methods. The source codes are available on https://github.com/P-N-Suganthan/CODES. [ABSTRACT FROM AUTHOR]

Forecasts of the wave heights can assist in the data-driven control of wave energy systems. However, the dynamic properties and extreme fluctuations of the historical observations pose challenges to the construction of forecasting models. This paper proposes a novel dynamic ensemble deep Echo state networks (ESN) to learn the dynamic characteristics of the significant wave height. The dynamic ensemble ESN creates a profound representation of the input and trains an independent readout module for each reservoir. To begin, numerous reservoir layers are built in a hierarchical order, adopting a reservoir pruning approach to filter out the poorer representations. Finally, a dynamic ensemble block is used to integrate the forecasts of all readout layers. The suggested model has been tested on twelve available datasets and statistically outperforms state-of-the-art approaches. • A dynamic ensemble deep echo state network is proposed. • A reservoir pruning strategy is proposed to remove inferior features. • An extensive review of significant wave height forecasting is conducted. • Our model manifests better performance than the SOTA models. [ABSTRACT FROM AUTHOR]

Electric load forecasting is essential for the planning and maintenance of power systems. However, its un-stationary and non-linear properties impose significant difficulties in predicting future demand. This paper proposes a novel ensemble deep Random Vector Functional Link (edRVFL) network for electricity load forecasting. The weights of hidden layers are randomly initialized and fixed during the training process. The hidden layers are stacked to enforce deep representation learning. Then, the model generates the forecasts using the ensemble of the outputs of each layer. Moreover, we also propose to augment the random enhancement features by empirical wavelet transformation (EWT). The raw load data are decomposed by EWT in a walk-forward approach without introducing future data leakage problems in the decomposition process. Finally, all the sub-series generated by the EWT, including raw data, are fed into the edRVFL for forecasting purposes. The proposed model is evaluated on sixteen publicly available time series from the Australian Energy Market Operator of the year 2020. The simulation results demonstrate the proposed model's superior performance over eleven forecasting methods in two error metrics and statistical tests on electricity load forecasting tasks. • An ensemble deep randomized network for forecasting is proposed. • Empirical wavelet transformation is used to augment unsupervised features. • A layer-wise tuning algorithm is proposed to optimize the structures. • Two error metrics and statistical tests are utilized to compare the models. [ABSTRACT FROM AUTHOR]

Emotion recognition based on electroencephalogram (EEG) signals is helpful in various fields, including medical healthcare. One possible medical application is to diagnose emotional disorders in patients. Humans tend to work and communicate efficiently when in a good mood. On the other hand, negative emotions can harm physical and mental health. Traditional EEG-based methods usually extract time-domain and frequency-domain features before classifying them. Convolutional Neural Networks (CNN) enables us to extract features and classify them end-to-end. However, most CNN methods use backpropagation to train their models, which can be computationally expensive, primarily when a complex model is used. Inspired by the successes of Random Vector Functional Link and Convolutional Random Vector Functional Link, we propose using a randomized CNN model for emotion recognition that removes the need for a backpropagation method. Also, we expand our randomized CNN method to a deep and ensemble version to improve emotion recognition performance. We do experiments on the commonly used publicly available Database for Emotion Analysis using the Physiological Signals (DEAP) dataset to evaluate our randomized CNN models. Results on the DEAP dataset show our models outperform all other models, with at least 95% accuracy for all subjects. Our ensemble version outperforms our shallow version, winning the shallow version in most subjects. • A randomized CNN is proposed to classify EEG signals. • Randomized CNN are expanded into deep and ensemble version for improved performance. • Randomized CNN outperforms other EEG-based models. • Deep and ensemble version substantially outperforms shallow Randomized CNN. [ABSTRACT FROM AUTHOR]

The lack of intuitive and active human–robot interaction makes it difficult to use upper-limb-assistive devices. In this paper, we propose a novel learning-based controller that intuitively uses onset motion to predict the desired end-point position for an assistive robot. A multi-modal sensing system comprising inertial measurement units (IMUs), electromyographic (EMG) sensors, and mechanomyography (MMG) sensors was implemented. This system was used to acquire kinematic and physiological signals during reaching and placing tasks performed by five healthy subjects. The onset motion data of each motion trial were extracted to input into traditional regression models and deep learning models for training and testing. The models can predict the position of the hand in planar space, which is the reference position for low-level position controllers. The results show that using IMU sensor with the proposed prediction model is sufficient for motion intention detection, which can provide almost the same prediction performance compared with adding EMG or MMG. Additionally, recurrent neural network (RNN)-based models can predict target positions over a short onset time window for reaching motions and are suitable for predicting targets over a longer horizon for placing tasks. This study's detailed analysis can improve the usability of the assistive/rehabilitation robots. [ABSTRACT FROM AUTHOR]

General deep learning methods achieve high-level semantic feature representation by aggregating hierarchical features, which performs well in object detection tasks. However, issues arise with general deep learning methods in UAV-based remote sensing image object detection tasks. Firstly, general feature aggregation methods such as stride convolution may lead to information loss in input samples. Secondly, common FPN methods introduce conflicting information by directly fusing feature maps from different levels. These shortcomings limit the model's detection performance on small and weak targets in remote sensing images. In response to these concerns, we propose an unmanned aerial vehicle (UAV) object detection algorithm, IF-YOLO. Specifically, our algorithm leverages the Information-Preserving Feature Aggregation (IPFA) module to construct semantic feature representations while preserving the intrinsic features of small objects. Furthermore, to filter out irrelevant information introduced by direct fusion, we introduce the Conflict Information Suppression Feature Fusion Module (CSFM) to improve the feature fusion approach. Additionally, the Fine-Grained Aggregation Feature Pyramid Network (FGAFPN) facilitates interaction between feature maps at different levels, reducing the generation of conflicting information during multi-scale feature fusion. The experimental results on the VisDrone2019 dataset demonstrate that in contrast to the standard YOLOv8-s, our enhanced algorithm achieves a mean average precision (mAP) of 47.3%, with precision and recall rates enhanced by 6.3% and 5.6%, respectively. [ABSTRACT FROM AUTHOR]

With the advancement of maritime unmanned aerial vehicles (UAVs) and deep learning technologies, the application of UAV-based object detection has become increasingly significant in the fields of maritime industry and ocean engineering. Endowed with intelligent sensing capabilities, the maritime UAVs enable effective and efficient maritime surveillance. To further promote the development of maritime UAV-based object detection, this paper provides a comprehensive review of challenges, relative methods, and UAV aerial datasets. Specifically, in this work, we first briefly summarize four challenges for object detection on maritime UAVs, i.e., object feature diversity, device limitation, maritime environment variability, and dataset scarcity. We then focus on computational methods to improve maritime UAV-based object detection performance in terms of scale-aware, small object detection, view-aware, rotated object detection, lightweight methods, and others. Next, we review the UAV aerial image/video datasets and propose a maritime UAV aerial dataset named MS2ship for ship detection. Furthermore, we conduct a series of experiments to present the performance evaluation and robustness analysis of object detection methods on maritime datasets. Eventually, we give the discussion and outlook on future works for maritime UAV-based object detection. The MS2ship dataset is available at https://github.com/zcj234/MS2ship. [ABSTRACT FROM AUTHOR]

Deep learning and fuzzy models provide powerful and practical techniques for solving large-scale deep-learning tasks. The fusion technique on deep learning and fuzzy system are generally classified into ensemble and integrated modes and materializes in information fusion, model fusion, and feature fusion. In an ensemble-based fusion, the fuzzy model either acts as an activation function or is operated as a separate process aggregating/preprocessing the information. Some early attempts in the field have successfully fused deep neural networks and fuzzy modeling concepts in ensemble mode. However, no effective attempts were made to fuse fuzzy models as an integrated feature-level fusion learning with graph neural networks (GNNs). This is mainly due to two challenges related to this fusion: (1) the number of fuzzy rules grows exponentially with the number of features that causes computational inefficiency, and (2) the solution space created by this fusion of fuzzy rules becomes complex due to multiple regression relations between inputs and outputs. Additionally, a simple linear regression at the output space would not be sufficient to model deep learning tasks. Therefore, this paper addresses these challenges by proposing a feature-level fusion method to fuse deep learning and fuzzy modeling where the latter technique is for integrated feature learning, called fuzzy forest graph neural network (FuzzyGNN), which creates a fuzzy learning forest fusing the linear graph transformers for deep learning tasks. We conducted experiments on fourteen machine learning datasets to test and validate the efficiency of the proposed FuzzyGNN model. Compared to state-of-the-art methods, our algorithm achieves the best results on four out of five machine learning datasets. The source code will be available at https://github.com/lingping-fuzzy/ , https://github.com/vojha-code and https://github.com/P-N-Suganthan. • We present an integrated fusion method for large-scale dataset classification. • We propose an integrated fusion of fuzzy learning and graph neural networks. • We propose sampling methods to address fuzzy rules exponentially explosion problem. • We propose adopting polynomial functions to simulate fitting prediction. • We experimentally justify the performance of the proposed model. [ABSTRACT FROM AUTHOR]