Search

Randomized neural networks (NNs), such as random vector functional link (RVFL) and extreme learning machine (ELM), have been widely applied in various classification problems owing to their computational efficiency and universal approximation capability. However, such approaches are designed for regular Euclidean data and lack the ability to generalize to complex structured data. Moreover, their randomly generated parameters often lead to a suboptimal decision boundary with a growing requirement of hidden neurons. In this article, we first propose a plain framework, termed randomized graph convolutional networks (RGCNs), to generalize the classic randomized NNs to the non-Euclidean domain. Then, a hybrid framework called evolution-driven RGCN (EvoRGCN) is presented by using adaptive differential evolution with novelty search strategy to seek the globally optimal graph embedding for the plain RGCN. Finally, we recast the classic ELM and RVFL under the proposed frameworks, resulting in four novel semi-supervised models, including the plain models [i.e., graph convolutional extreme learning machines (GCELMs) and graph convolutional RVFL (GCRVFL)] and the optimized models (i.e., O-GCELM and O-GCRVFL). We show that our approaches are the natural generalization of the traditional randomized NNs in the non-Euclidean domain. Furthermore, our approaches not only retain the advantages of the classic approaches but also enable them to handle graph data. We compare our approaches against many existing methods across regular datasets and graph benchmarks, demonstrating that the proposed approaches dramatically outperform the compared methods with better generalization ability and robustness. Particularly, we quantitatively show the performance ranking of different randomized NNs, i.e., O-GCRVFL $> $ O-GCELM $\approx $ GCRVFL $> $ GCELM $\approx $ RVFL $> $ ELM. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) is characterized by simplicity, generalization ability, and computational efficiency. However, previous ELMs fail to consider the inherent high-order relationship among data points, resulting in being powerless on structured data and poor robustness on noise data. This paper presents a novel semi-supervised ELM, termed Hypergraph Convolutional ELM (HGCELM), based on using hypergraph convolution to extend ELM into the non-Euclidean domain. The method inherits all the advantages from ELM, and consists of a random hypergraph convolutional layer followed by a hypergraph convolutional regression layer, enabling it to model complex intraclass variations. We show that the traditional ELM is a special case of the HGCELM model in the regular Euclidean domain. Extensive experimental results show that HGCELM remarkably outperforms eight competitive methods on 26 classification benchmarks. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) has gained lots of research interests due to its universal approximation capability and fast learning speed. Although several prior works have focused on developing deep ELM, it is still an open problem to design effective deep ELM. Stacking random layers will result in overfitting and accumulation of random errors. To address this issue, this paper presents a simple yet effective deep ELM called Densely Connected Convolutional ELM (DC2ELM) for hyperspectral image spectral-spatial classification. First, we introduce dense connection into ELM to make full use of intermediate feature maps produced by randomized convolutional layers, which is beneficial to reduce the random error. Secondly, stacked ELM auto-encoders are employed to generate reduced representation, leading to a deeper architecture. The proposed approach consists of fewer trainable parameters than traditional convolutional neural networks and can easily be trained without any iterative parameters tuning, making it easier to implement and apply in practice. We compare the proposed approach with many prior arts over three real hyperspectral images, showing that the proposed approach can achieve superior performance using limited training data and with a reduced risk of overfitting the training data. [ABSTRACT FROM AUTHOR]

Currently, deep neural networks (DNNs) are an important method for handling hyperspectral image (HSI) classification because of their good performance in image processing. However, DNNs’ performance depends on a massive number of training data and hyperparameters that are carefully fine-tuned, which results in structural complexity and a time-consuming process. Deep forest is a novel deep learning method that does not need much training data and has a simple structure. In this paper, we first design a deep forest for spectral-based HSI classification and then propose an improved deep forest algorithm, named deep multigrained cascade forest (dgcForest), for spatial-based HSI classification. On the one hand, the cascade forest in dgcForest is used in multigrained scanning, which enhances the performance; on the other hand, a pooling layer is added after the multigrained scanning to reduce the output dimensions. To demonstrate that our proposed algorithm presents a good performance in HSI classification, we analyze the hyperparameters of deep forest and dgcForest and compare them with other methods on the biased and unbiased data sets, which illustrates that our method is superior to other state-of-the-art deep learning methods. [ABSTRACT FROM AUTHOR]

Highlights • A novel hierarchical ensemble of ELM integrated with representation learning and ensemble learning. • Deep cascade structure is used for re-representing features. • Sparse connection and feature bagging are used to encourage individual diversity. • HE-ELM significantly outperforms many existing ensemble and representation learning methods. Abstract Extreme Learning Machine (ELM), which is proposed for generalized single-hidden layer feedforward neural networks, has become a popular research topic due to its unique characteristics. However, the random nature inherent in ELM's hidden layer results in unstable performance and a large number of hidden neurons is required, making the risk of overfitting increased. In this paper, we propose a simple but effective ensemble approach, called Hierarchical Ensemble of Extreme Learning Machine (HE-ELM), to improve ELM. To encourage the diversity of component ELMs, two strategies are taken into account, namely, the sparse connection to component ELMs and feature bagging. The resulting architecture is able to integrate both representation learning and ensemble learning with relatively fewer parameters and consists of independent component ELMs, making it easy to implement, train, and apply in practice. We compare results of the proposed HE-ELM with existing methods for 22 classification problems, showing that HE-ELM is able to achieve significant improvement in terms of classification accuracy, with a reduced risk of overfitting the training data. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) has been widely used for various classification problems. However, the traditional ELMs are typically based on the regular Euclidean data, thus ignoring the intrinsic structured information among data and resulting in poor robustness. In this paper, we present a novel semi-supervised learning framework, termed Graph Convolutional Extreme Learning Machines (GCELM), based on extending the traditional ELM to the non-Euclidean domain. Technically, we recast ELM layers into a randomized graph convolutional embedding layer followed by a graph convolutional regression layer, which endows ELM with the capability to handle graphs in the non-Euclidean domain. Benefiting from the diversity of the randomized graph convolution, we further propose an enhanced GCELM, i.e., Voting-based GCELM (V-GCELM), by using a simple voting ensemble strategy. The proposed methods preserve the advantages of ELMs, thus being more efficient than the gradient-based graph convolutional networks but without loss of graph learning ability. Extensive experiments on 36 benchmark datasets demonstrate that the proposed methods significantly outperform many previous semi-supervised classification methods. • Classic ELM is generalized into the non-Euclidean domain by using graph convolution. • An ensemble-based graph convolutional ELM is proposed for semi-supervised learning. • The proposed methods are benchmarked on 36 standard UCI datasets. • Our methods significantly outperform many existing methods by a large margin. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) consists of hundreds of narrow spectral band components with rich spectral and spatial information. Extreme Learning Machine (ELM) has been widely used for HSI analysis. However, the classical ELM is difficult to use for sparse feature leaning due to its randomly generated hidden layer. In this paper, we propose a novel unsupervised sparse feature learning approach, called Evolutionary Multiobjective-based ELM (EMO-ELM), and apply it to HSI feature extraction. Specifically, we represent the task of constructing the ELM Autoencoder (ELM-AE) as a multiobjective optimization problem that takes the sparsity of hidden layer outputs and the reconstruction error as two conflicting objectives. Then, we adopt an Evolutionary Multiobjective Optimization (EMO) method to solve the two objectives, simultaneously. To find the best solution from the Pareto solution set and construct the best trade-off feature extractor, a curvature-based method is proposed to focus on the knee area of the Pareto solutions. Benefited from the EMO, the proposed EMO-ELM is less prone to fall into a local minimum and has fewer trainable parameters than gradient-based AEs. Experiments on two real HSIs demonstrate that the features learned by EMO-ELM not only preserve better sparsity but also achieve superior separability than many existing feature learning methods. [ABSTRACT FROM AUTHOR]

Highlights • We propose a multiobjective optimization-based sparse extreme learning machine (MO-SELM) for classification and regression tasks. • The sparse connecting structure of ELM is designed for learning a more compact network. • We improve MOEA/D to optimize the proposed multiobjective model and make decisions by ensemble learning. • Experimental results reveal the superior performance of the proposed MO-SELM. Abstract Extreme Learning Machine (ELM) is a popular machine learning method and has been widely applied to real-world problems due to its fast training speed and good generalization performance. However, in ELM, the randomly assigned input weights and hidden biases usually degrade the generalization performance. Furthermore, ELM is considered as an empirical risk minimization model and easily leads to overfitting when dataset exists some outliers. In this paper, we proposed a novel algorithm named Multiobjective Optimization-based Sparse Extreme Learning Machine (MO-SELM), where parameter optimization and structure learning are integrated into the learning process to simultaneously enhance the generalization performance and alleviate the overfitting problem. In MO-SELM, the training error and the connecting sparsity are taken as two conflicting objectives of the multiobjective model, which aims to find sparse connecting structures with optimal weights and biases. Then, a hybrid encoding-based MOEA/D is used to optimize the multiobjective model. In addition, ensemble learning is embedded into this algorithm to make decisions after multiobjective optimization. Experimental results of several classification and regression applications demonstrate the effectiveness of the proposed MO-SELM. [ABSTRACT FROM AUTHOR]