Search

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

• Multinomail distribution is used for improving Sparse Bayesian Extreme Learning Machine. • The proposed method is an effective sparse Bayesian learning for multiclass classification. • Extensive experiments on real-world benchmark datasets verified the effectiveness of the proposed method. Sparse Bayesian extreme learning machine (SBELM) is a probabilistic model with three-layer neural network, which is superior to extreme learning machine (ELM) in model generalization, sparsity and execution time. In SBELM, Bernoulli distribution is employed for binary classification, and then extended to multi-class classification using pairwise coupling. However, pairwise coupling suffers from three significant drawbacks for multi-class classification: 1) classification ambiguity and uncovered class regions; 2) large model size; 3) insufficient uncertainty representation for label prediction in probabilities. To alleviate these drawbacks, multinomial Bayesian extreme learning machine (MBELM) is proposed that employs multinomial distribution, which is proposed for multi-class classification. For the sake of various concerns between sparsity and accuracy, two sparse mechanisms namely automatic relevance determination (ARD) and L 1 penalty are respectively integrated with MBELM. The experimental results show that, compared to SBELM, the proposed MBELM improves the test accuracy and the model size respectively up to 5% better, and 94 times smaller for multi-class classification. [ABSTRACT FROM AUTHOR]