Search

A teaching-learning-based optimization algorithm (TLBO) which uses a variable population size in the form of a triangle form (VTTLBO) is proposed in the paper. The main goal of the proposed method is to decrease the computing cost of original TLBO and extend it for optimizing the parameters of artificial neural network (ANN). In the proposed algorithm, the evolutionary process is divided into some equal periods according to the maximal generation. The population size in each period is changed in form of triangle. In the linear increasing phase of population’s number, some new individuals are generated with gauss distribution by using the adaptive mean and variance of the population. In the linear decreasing phase of population׳s number, some highly similar individuals are deleted. To compare the performance of the proposed method with some other methods, Saw-tooth teaching-learning-based optimization algorithm is also designed with simulating the basic principle of Saw-tooth genetic algorithm (STGA), and some other EAs with the fixed population size are also simulated. A variety of benchmark problems and system modeling and prediction problems with ANN are tested in this paper, the results indicate that the computation cost of the given method is small and the convergence accuracy and speed of it are high. [ABSTRACT FROM AUTHOR]

Dendrite morphological neurons are a type of artificial neural network that works with min and max operators instead of algebraic products. These morphological operators build hyperboxes in N -dimensional space. These hyperboxes allow the proposal of training methods based on heuristics without using an optimisation method. In literature, it has been claimed that these heuristic-based trainings have advantages: there are no convergence problems, perfect classification can always be reached and training is performed in only one epoch. In this paper, we show that these assumed advantages come with a cost: these heuristics increase classification errors in the test set because they are not optimal and learning generalisation is poor. To solve these problems, we introduce a novel method to train dendrite morphological neurons based on stochastic gradient descent for classification tasks, using these heuristics just for initialisation of learning parameters. Experiments show that we can enhance the testing error in comparison with solely heuristic-based training methods. This approach can reach competitive performance with respect to other popular machine learning algorithms. [ABSTRACT FROM AUTHOR]

Distinct from conventional approaches to generic vehicle type recognition, this paper proposes an approach to fine-grained vehicle type recognition that leverages multiple cues jointly to convolutional neural networks (CNNs). Fine-grained vehicle recognition is inherently difficult with many challenges, including (i) how to incorporate multiple cues through joint optimization instead of separately handling, (ii) how to learn data-driven representation for vehicles instead of hand-crafted features and (iii) how to perceive and localize the vehicle region instead of using the whole image. Our multi-task CNNs localize vehicles in the first stage and recognize subclasses in the second stage, allowing us to handle samples with cluttered backgrounds and those where vehicles do not fill most of the image. Through collaborative feature learning from multiple tasks in each stage, our approach can handle subtle inter-class variations at the subordinate level. We also develop new vehicle-oriented data augmentation strategies in CNN training. To advance research on vehicle-centered tasks, we release a new vehicle dataset that provides both semantic labels and bounding boxes. This dataset can be used in vehicle localization, recognition, viewpoint estimation, and so on. Extensive experiments on two benchmark vehicle datasets demonstrate that our approach outperforms state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Similarity-preserving hashing is a widely used method for nearest neighbor search in large-scale image retrieval. Recently, supervised hashing methods are appealing in that they learn compact hash codes with fewer bits by incorporating supervised information. In this paper, we propose a new two-stage supervised hashing methods which decomposes the hash learning process into a stage of learning approximate hash codes followed by a stage of learning hash functions. In the first stage, we propose a margin-based objective to find approximate hash codes such that a pair of hash codes associating to a pair of similar (dissimilar) images has sufficiently small (large) Hamming distance. This objective results in a challenging optimization problem. We develop a coordinate descent algorithm to efficiently solve this optimization problem. In the second stage, we use convolutional neural networks to learn hash functions. We conduct extensive evaluations on several benchmark datasets with different kinds of images. The results show that the proposed margin-based hashing method has substantial improvement upon the state-of-the-art supervised or unsupervised hashing methods. [ABSTRACT FROM AUTHOR]

This paper proposes an improved stochastic second order learning algorithm for supervised neural network training. The proposed algorithm, named bounded stochastic diagonal Levenberg–Marquardt (B-SDLM), utilizes both gradient and curvature information to achieve fast convergence while requiring only minimal computational overhead than the stochastic gradient descent (SGD) method. B-SDLM has only a single hyperparameter as opposed to most other learning algorithms that suffer from the hyperparameter overfitting problem due to having more hyperparameters to be tuned. Experiments using the multilayer perceptron (MLP) and convolutional neural network (CNN) models have shown that B-SDLM outperforms other learning algorithms with regard to the classification accuracies and computational efficiency (about 5.3% faster than SGD on the mnist-rot-bg-img database). It can classify all testing samples correctly on the face recognition case study based on AR Purdue database. In addition, experiments on handwritten digit classification case studies show that significant improvements of 19.6% on MNIST database and 17.5% on mnist-rot-bg-img database can be achieved in terms of the testing misclassification error rates (MCRs). The computationally expensive Hessian calculations are kept to a minimum by using just 0.05% of the training samples in its estimation or updating the learning rates once per two training epochs, while maintaining or even achieving lower testing MCRs. It is also shown that B-SDLM works well in the mini-batch learning mode, and we are able to achieve 3.32 × performance speedup when deploying the proposed algorithm in a distributed learning environment with a quad-core processor. [ABSTRACT FROM AUTHOR]

Multi-instance learning (MIL) is one of promising paradigms in the supervised learning aiming to handle real world classification problems where a classification target contains several featured sections, e.g., an image typically contains several salient regions. In this paper, we propose a highly efficient learning method for MI classification based on hierarchical extreme learning machine (ELM), called MI-ELM. Specifically, a double-hidden layers feedforward network (DLFN) is designed to serve as the MI classifier. Then, the MI classification is formulated as an optimization problem. Moreover, the output weights of DLFN can be analytically determined by solving the aforementioned optimization problem. The merits of MI-ELM are as follows: (i) MI-ELM extends the single-layer ELM to be a hierarchical one that well fits for training DLFNs in MI classification. (ii) The input and hidden-layer parameters of DLFNs are assigned randomly rather than tuned iteratively, and the output weights of DLFNs can be determined analytically in one step. Therefore, MI-ELM significantly enhances the efficiency of the DLFN without notable loss of the classification accuracy. Experimental results over several real-world data sets demonstrate that the proposed MI-ELM method significantly outperforms existing kernel methods for MI classification in terms of the classification accuracy and the classification time. [ABSTRACT FROM AUTHOR]

In this paper, three novel classification algorithms aiming at (semi-)supervised action classification are proposed. Inspired by the effectiveness of discriminant subspace learning techniques and the fast and efficient Extreme Learning Machine (ELM) algorithm for Single-hidden Layer Feedforward Neural networks training, the ELM algorithm is extended by incorporating discrimination criteria in its optimization process, in order to enhance its classification performance. The proposed Discriminant ELM algorithm is extended, by incorporating proper regularization in its optimization process, in order to exploit information appearing in both labeled and unlabeled action instances. An iterative optimization scheme is proposed in order to address multi-view action classification. The proposed classification algorithms are evaluated on three publicly available action recognition databases providing state-of-the-art performance in all the cases. [ABSTRACT FROM AUTHOR]

Abstract: This paper proposes a hierarchical multi-dimensional differential evolution (HMDDE) algorithm, which is an automatic computational frame work for the optimization of beta basis function neural network (BBFNN) wherein the neural network architecture, weights connection, learning algorithm and its parameters are adapted according to the problem. In the HMDDE-designed neural network, the number of individuals of the population multi-dimensions is the number of beta neural networks. The population of HMDDE forms multiple beta networks with different structures at the higher level and each individual of the previous population is optimized at a lower hierarchical level to improve the performance of each individual. For the beta neural network consisting of m neurons, n individuals (different lengths) are formed in the upper level to optimize the structure of the beta neural network. In the lower level, the population within the same length is to optimize the free parameters of the beta neural network. To evaluate the comparative performance, we used benchmark problems drawn from identification system and time series prediction area. Empirical results illustrate that the HMDDE produces a better generalization performance. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce an advanced architecture of K-means clustering-based polynomial Radial Basis Function Neural Networks (p-RBF NNs) designed with the aid of Particle Swarm Optimization (PSO) and Differential Evolution (DE) and develop a comprehensive design methodology supporting their construction. The architecture of the p-RBF NNs comes as a result of a synergistic usage of the evolutionary optimization-driven hybrid tools. The connections (weights) of the proposed p-RBF NNs being of a certain functional character and are realized by considering four types of polynomials. In order to design the optimized p-RBF NNs, a prototype (center value) of each receptive field is determined by running the K-means clustering algorithm and then a prototype and a spread of the corresponding receptive field are further optimized through running Particle Swarm Optimization (PSO) and Differential Evolution (DE). The Weighted Least Square Estimation (WLSE) is used to estimate the coefficients of the polynomials (which serve as functional connections of the network). The performance of the proposed model and the comparative analysis involving models designed with the aid of PSO and DE are presented in case of a nonlinear function and two Machine Learning (ML) datasets [Copyright &y& Elsevier]

Abstract: In this paper, we propose a methodology for training a new model of artificial neural network called the generalized radial basis function (GRBF) neural network. This model is based on generalized Gaussian distribution, which parametrizes the Gaussian distribution by adding a new parameter . The generalized radial basis function allows different radial basis functions to be represented by updating the new parameter . For example, when GRBF takes a value of , it represents the standard Gaussian radial basis function. The model parameters are optimized through a modified version of the extreme learning machine (ELM) algorithm. In the methodology proposed (MELM-GRBF), the centers of each GRBF were taken randomly from the patterns of the training set and the radius and values were determined analytically, taking into account that the model must fulfil two constraints: locality and coverage. An thorough experimental study is presented to test its overall performance. Fifteen datasets were considered, including binary and multi-class problems, all of them taken from the UCI repository. The MELM-GRBF was compared to ELM with sigmoidal, hard-limit, triangular basis and radial basis functions in the hidden layer and to the ELM-RBF methodology proposed by Huang et al. (2004) . The MELM-GRBF obtained better results in accuracy than the corresponding sigmoidal, hard-limit, triangular basis and radial basis functions for almost all datasets, producing the highest mean accuracy rank when compared with these other basis functions for all datasets. [Copyright &y& Elsevier]

Abstract: The design of radial basis function neural networks (RBFNNs) still remains as a difficult task when they are applied to classification or to regression problems. The difficulty arises when the parameters that define an RBFNN have to be set, these are: the number of RBFs, the position of their centers and the length of their radii. Another issue that has to be faced when applying these models to real world applications is to select the variables that the RBFNN will use as inputs. The literature presents several methodologies to perform these two tasks separately, however, due to the intrinsic parallelism of the genetic algorithms, a parallel implementation will allow the algorithm proposed in this paper to evolve solutions for both problems at the same time. The parallelization of the algorithm not only consists in the evolution of the two problems but in the specialization of the crossover and mutation operators in order to evolve the different elements to be optimized when designing RBFNNs. The subjacent genetic algorithm is the non-sorting dominated genetic algorithm II (NSGA-II) that helps to keep a balance between the size of the network and its approximation accuracy in order to avoid overfitted networks. Another of the novelties of the proposed algorithm is the incorporation of local search algorithms in three stages of the algorithm: initialization of the population, evolution of the individuals and final optimization of the Pareto front. The initialization of the individuals is performed hybridizing clustering techniques with the mutual information (MI) theory to select the input variables. As the experiments will show, the synergy of the different paradigms and techniques combined by the presented algorithm allow to obtain very accurate models using the most significant input variables. [Copyright &y& Elsevier]