Search

There has been some improvement in the treatment of preterm infants, which has helped to increase their chance of survival. However, the rate of premature births is still globally increasing. As a result, this group of infants is most at risk of developing severe medical conditions that can affect the respiratory, gastrointestinal, immune, central nervous, auditory and visual systems. There is a strong body of evidence emerging that suggests the analysis of uterine electrical signals, from the abdominal surface (Electrohysterography – EHG), could provide a viable way of diagnosing true labour and even predict preterm deliveries. This paper explores this idea further and presents a new dynamic self-organized network immune algorithm that classifies term and preterm records, using an open dataset containing 300 records (38 preterm and 262 term ). Using the dataset, oversampling and cross validation techniques are evaluated against other similar studies. The proposed approach shows an improvement on existing studies with 89% sensitivity , 91% specificity , 90% positive predicted value , 90% negative predicted value , and an overall accuracy of 90%. [ABSTRACT FROM AUTHOR]

Abstract: The Growing Neural Gas algorithm (GNG) is a well-known classification algorithm that is capable of capturing topological relationships that exist in the input data. Unfortunately, simple implementations of the GNG algorithm have time complexity , where n is the number of nodes in the graph. This fact makes these implementations impractical for use in production environments where large data sets are used. This paper aims to propose an optimized implementation that breaks the barrier and that addresses data in high-dimensional spaces without changing the GNG semantics. The experimental results show speedups of over 50 times for graphs with 200,000 nodes. [Copyright &y& Elsevier]

Abstract: Breast tumor detection in digital mammography is one of the most important methods of breast cancer prevention. Computer-aided diagnosis (CAD) based on extreme learning machine (ELM) has significant meanings for breast tumor detection as it has good generalization abilities and a high learning efficiency. In this paper, a breast tumor detection algorithm in digital mammography based on ELM is proposed. First, a median filter is used for noise reduction, and contrast enhancement of the digital mammography in data preprocessing. Next, methods of wavelet modulus maxima transform, morphological operation and region growth are used for the breast tumor edge segmentation. Then, five textural features and five morphological features are extracted. Finally, an ELM classifier is used to detect the breast tumor. Comparing breast tumor detection based on Support Vector Machines (SVM), with breast tumor detection based on ELM, not only does ELM have a better classification accuracy than SVM, but it also has a greatly improved training speed. [Copyright &y& Elsevier]

Abstract: Traffic sign recognition is an important and active research topic of intelligent transport system. With a constant increasing of the training database size, not only the recognition accuracy, but also the computation complexity should be considered in designing a feasible recognition approach. In this paper, an effective and efficient algorithm based on a relatively new artificial neural network, extreme learning machine (ELM), is proposed for traffic sign recognition. In the proposed algorithm, the locally normalized histograms of the oriented gradient (HOG) descriptors, which are extracted from the traffic sign images, are used as the features and the inputs of the ELM classification model. Moreover, the ratio of feature's between-category to within-category sums of squares (BW) is designed as a feature selection criterion to improve the recognition accuracy and to decrease the computation burden. Application on a well known database, German traffic sign recognition benchmark (GTSRB) dataset, demonstrates the feasibility and efficiency of the proposed BW-ELM model. [Copyright &y& Elsevier]

Abstract: Extreme learning machine (ELM), used for the “generalized” single-hidden-layer feedforward networks (SLFNs), is a unified learning platform that can use a widespread type of feature mappings. In theory, ELM can approximate any target continuous function and classify any disjoint regions; in application, many experiment results have already demonstrated the good performance of ELM. In view of the good properties of the ELM feature mapping, the clustering problem using ELM feature mapping techniques is studied in this paper. Experiments show that the proposed ELM kMeans algorithm and ELM NMF (nonnegative matrix factorization) clustering can get better clustering results than the corresponding Mercer kernel based methods and the traditional algorithms using the original data. Moreover, the proposed methods have the advantage of being more convenient to implementation and computation, as the ELM feature mapping is much simpler than the Mercer kernel function based feature mapping methods. [Copyright &y& Elsevier]

Abstract: In this paper, a novel 1-norm extreme learning machine (ELM) for regression and multiclass classification is proposed as a linear programming problem whose solution is obtained by solving its dual exterior penalty problem as an unconstrained minimization problem using a fast Newton method. The algorithm converges from any starting point and can be easily implemented in MATLAB. The main advantage of the proposed approach is that it leads to a sparse model representation meaning that many components of the optimal solution vector will become zero and therefore the decision function can be determined using much less number of hidden nodes in comparison to ELM. Numerical experiments were performed on a number of interesting real-world benchmark datasets and their results are compared with ELM using additive and radial basis function (RBF) hidden nodes, optimally pruned ELM (OP-ELM) and support vector machine (SVM) methods. Similar or better generalization performance of the proposed method on the test data over ELM, OP-ELM and SVM clearly illustrates its applicability and usefulness. [Copyright &y& Elsevier]

Abstract: Regularized discriminant analysis (RDA) and its special case uncorrelated linear discriminant analysis (ULDA) are important subspace learning methods proposed recently to handle the small sample size (SSS) problem of linear discriminant analysis (LDA). One important unsolved issue of RDA is how to automatically determine an appropriate regularization parameter without resorting to unscalable procedures like cross-validation (CV). In this paper, we develop a novel efficient algorithm to automatically estimate the regularization parameter based on a geometric interpretation of RDA. We further provide a formal analysis of the proposed method, and show that it is robust to the perturbation in the feature space of the training data. The extensive experiments on various benchmark datasets verify the scalability and effectiveness of our approach, compared with the state-of-the-art algorithms. [Copyright &y& Elsevier]

Abstract: In this paper we propose a novel data clustering algorithm based on the idea of considering the individual data items as cells belonging to a uni-dimensional cellular automaton. Our proposed algorithm combines insights into both social segregation models based on Cellular Automata Theory, where the data items themselves are able to move autonomously in lattices, and also from Ants Clustering algorithms, particularly in the idea of distributing at random the data items to be clustered in lattices. We also consider an automatic method for determining the number of clusters in the dataset by analyzing the intra-cluster variances. A series of experiments with both synthetic and real datasets are presented in order to study empirically the convergence and performance results. These experimental results are compared to the obtained by conventional clustering algorithms. [Copyright &y& Elsevier]

Abstract: This paper proposes the fusion of lattice independent component analysis (LICA) features with linear features obtained from conventional methods. Specifically, we compute class conditional LICA, where separate endmembers are extracted from each class dataset. We find that this fusion approach improves systematically the recognition accuracy in face recognition applications. We report experimental results using seven state-of-the-art linear feature extraction algorithms on four public face databases using Extreme Learning Machines (ELMs) for the classification building algorithm. [Copyright &y& Elsevier]

Abstract: In a previous work (Mohemmed et al., Method for training a spiking neuron to associate input–output spike trains) [1] we have proposed a supervised learning algorithm based on temporal coding to train a spiking neuron to associate input spatiotemporal spike patterns to desired output spike patterns. The algorithm is based on the conversion of spike trains into analogue signals and the application of the Widrow–Hoff learning rule. In this paper we present a mathematical formulation of the proposed learning rule. Furthermore, we extend the application of the algorithm to train a SNN consisting of multiple spiking neurons to perform spatiotemporal pattern classification and we show that the accuracy of classification is improved significantly over a single spiking neuron. We also investigate a number of possibilities to map the temporal output of the trained spiking neuron into a class label. Potential applications for motor control in neuro-rehabilitation and neuro-prosthetics are discussed as a future work. [Copyright &y& Elsevier]

Abstract: In this paper, we propose an algorithm entitled “partitioned OS-ELM” (POS-ELM) that partitions a large data matrix into small matrices, applies an RLS (Recursive Least Square) scheme in each of the small sub-matrices and assembles the whole estimation vector by the concatenation of the sub-vectors from the RLS outputs of the sub-matrices. Consequently, the algorithm is less complex than the conventional OS-ELM and maintains an almost compatible estimation performance. [Copyright &y& Elsevier]

Abstract: In design of a multibiometric system, a major concern is the learning cost in terms of computation complexity and memory usage due to large data size. In this paper, we propose an online learning network to circumvent the computational problem. Although conventional online learning algorithms can be adopted, their optimization of the fitting distance residuals does not meet the actual classification error requirement. A direct optimization to the classification performance is thus desired. Since the proposed classification-based formulation involves a class-specific weight which varies according to the total number of genuine-users and imposters, an online learning formulation becomes non-trivial. Extensive empirical evaluations on publicly available data sets show promising potential of the proposed method in terms of fusion verification accuracy and computational cost. [Copyright &y& Elsevier]

Abstract: The initial localized generalization error model (LGEM) aims to find an upper bound of error between a target function and a radial basis function neural network (RBFNN) within a neighborhood of the training samples. The contribution of LGEM can be briefly described as that the generalization error is less than or equal to the summation of three terms: training error, stochastic sensitivity measure (SSM), and a constant. This paper extends the initial LGEM to a new LGEM model for single-hidden layer feed-forward neural networks (SLFNs) trained with extreme learning machine (ELM) which is a type of new training algorithms without iterations. The development of this extended LGEM can provide some useful guidelines for improving the generalization ability of SLFNs trained with ELM. An algorithm for architecture selection of the SLFNs is also proposed based on the extended LGEM. Experimental results on a number of benchmark data sets show that an approximately optimal architecture in terms of number of neurons of a SLFN can be found using our method. Furthermore, the experimental results on eleven UCI data sets show that the proposed method is effective and efficient. [Copyright &y& Elsevier]

Abstract: Most of the existing methods for pedestrian detection work well, only when the following assumption is satisfied: the features extracted from the training dataset and the testing dataset have very similar distributions in the feature space. However, in practice, this assumption does not hold because of the scene complexity and variation. In this paper, a new method is proposed for detecting pedestrians in various scenes based on the transfer learning technique. Our proposed method employs the following two strategies for improving the pedestrian detection performance. First, a new sample screening method based on manifold learning is proposed. The basic idea is to choose samples from the training set, which may be similar to the samples from the unseen scene, and then merge the selected samples into the unseen set. Second, a new classification model based on transfer learning is proposed. The advantage of the classification model is that only a small number of samples need to be used from the unseen scenes. Most of the training samples are still obtained from the training scene, which take up to 90% of the entire training samples. Compared to the traditional pedestrian detection methods, the proposed algorithm can adapt to different scenes for detecting pedestrians. Experiments on two pedestrian detection benchmark datasets, DC and NICTA, showed that the method can obtain better performance as compared to other previous methods. [Copyright &y& Elsevier]

Abstract: Distance metric plays an important role in many machine learning tasks. The distance between samples is mostly measured with a predefined metric, ignoring how the samples distribute in the feature space and how the features are correlated. This paper proposes a semi-supervised distance metric learning method by exploring feature correlations. Specifically, unlabeled samples are used to calculate the prediction error by means of local linear regression. Labeled samples are used to learn discriminative ability, that is, maximizing the between-class covariance and minimizing the within-class covariance. We then fuse the knowledge learned from both labeled and unlabeled samples into an overall objective function which can be solved by maximum eigenvectors. Our algorithm explores both labeled and unlabeled information as well as data distribution. Experimental results demonstrates the superiority of our method over several existing algorithms. [Copyright &y& Elsevier]

Abstract: Recent research has shown the benefit of incorporating the radius of the Minimal Enclosing Ball (MEB) of training data into Multiple Kernel Learning (MKL). However, straightforwardly incorporating this radius leads to complex learning structure and considerably increased computation. Moreover, the notorious sensitivity of this radius to outliers can adversely affect MKL. In this paper, instead of directly incorporating the radius of MEB, we incorporate its close relative, the trace of data scattering matrix, to avoid the above problems. By analyzing the characteristics of the resulting optimization, we show that the benefit of incorporating the radius of MEB can be fully retained. More importantly, our algorithm can be effortlessly realized within the existing MKL framework such as SimpleMKL. The mere difference is the way to normalize the basic kernels. Although this kernel normalization is not our invention, our theoretic derivation uncovers why this normalization can achieve better classification performance, which has not appeared in the literature before. As experimentally demonstrated, our method achieves the overall best learning performance in various settings. In another perspective, our work improves SimpleMKL to utilize the information of the radius of MEB in an efficient and practical way. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a sequential learning algorithm for a neural network classifier based on human meta-cognitive learning principles. The network, referred to as Meta-cognitive Neural Network (McNN). McNN has two components, namely the cognitive component and the meta-cognitive component. A radial basis function network is the fundamental building block of the cognitive component. The meta-cognitive component controls the learning process in the cognitive component by deciding what-to-learn, when-to-learn and how-to-learn. When a sample is presented at the cognitive component of McNN, the meta-cognitive component chooses the best learning strategy for the sample using estimated class label, maximum hinge error, confidence of classifier and class-wise significance. Also sample overlapping conditions are considered in growth strategy for proper initialization of new hidden neurons. The performance of McNN classifier is evaluated using a set of benchmark classification problems from the UCI machine learning repository and two practical problems, viz., the acoustic emission for signal classification and a mammogram data set for cancer classification. The statistical comparison clearly indicates the superior performance of McNN over reported results in the literature. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce an advanced architecture of K-means clustering-based polynomial Radial Basis Function Neural Networks (p-RBF NNs) designed with the aid of Particle Swarm Optimization (PSO) and Differential Evolution (DE) and develop a comprehensive design methodology supporting their construction. The architecture of the p-RBF NNs comes as a result of a synergistic usage of the evolutionary optimization-driven hybrid tools. The connections (weights) of the proposed p-RBF NNs being of a certain functional character and are realized by considering four types of polynomials. In order to design the optimized p-RBF NNs, a prototype (center value) of each receptive field is determined by running the K-means clustering algorithm and then a prototype and a spread of the corresponding receptive field are further optimized through running Particle Swarm Optimization (PSO) and Differential Evolution (DE). The Weighted Least Square Estimation (WLSE) is used to estimate the coefficients of the polynomials (which serve as functional connections of the network). The performance of the proposed model and the comparative analysis involving models designed with the aid of PSO and DE are presented in case of a nonlinear function and two Machine Learning (ML) datasets [Copyright &y& Elsevier]

Abstract: Semi-supervised learning techniques have gained increasing attention in the machine learning community, as a result of two main factors: (1) the available data is exponentially increasing; (2) the task of data labeling is cumbersome and expensive, involving human experts in the process. In this paper, we propose a network-based semi-supervised learning method inspired by the modularity greedy algorithm, which was originally applied for unsupervised learning. Changes have been made in the process of modularity maximization in a way to adapt the model to propagate labels throughout the network. Furthermore, a network reduction technique is introduced, as well as an extensive analysis of its impact on the network. Computer simulations are performed for artificial and real-world databases, providing a numerical quantitative basis for the performance of the proposed method. [Copyright &y& Elsevier]

Abstract: In machine learning the so-called curse of dimensionality, pertinent to many classification algorithms, denotes the drastic increase in computational complexity and classification error with data having a great number of dimensions. In this context, feature selection techniques try to reduce dimensionality finding a new more compact representation of instances selecting the most informative features and removing redundant, irrelevant, and/or noisy features. In this paper, we propose a filter-based feature selection method for working in the multiple-instance learning scenario called ReliefF-MI; it is based on the principles of the well-known ReliefF algorithm. Different extensions are designed and implemented and their performance checked in multiple instance learning. ReliefF-MI is applied as a pre-processing step that is completely independent from the multi-instance classifier learning process and therefore is more efficient and generic than wrapper approaches proposed in this area. Experimental results on five benchmark real-world data sets and 17 classification algorithms confirm the utility and efficiency of this method, both statistically and from the point of view of execution time. [Copyright &y& Elsevier]

Abstract: This paper presents a distributed system for the recognition of human actions using views of the scene grabbed by different cameras. 2D frame descriptors are extracted for each available view to capture the variability in human motion. These descriptors are projected into a lower dimensional space and fed into a probabilistic classifier to output a posterior distribution of the action performed according to the descriptor computed at each camera. Classifier fusion algorithms are then used to merge the posterior distributions into a single distribution. The generated single posterior distribution is fed into a sequence classifier to make the final decision on the performed activity. The system can instantiate different algorithms for the different tasks, as the interfaces between modules are clearly defined. Results on the classification of the actions in the IXMAS dataset are reported. The accuracy of the proposed system is similar to state-of-the-art 3D methods, even though it uses only well-known 2D pattern recognition techniques and does not need to project the data into a 3D space or require camera calibration parameters. [Copyright &y& Elsevier]

Abstract: From the perspective of manifold learning, the weight between two nodes of graph plays an indispensable role, which provides the similarity between pairwise nodes, and can effectively reveal the intrinsic relationship between data classes. In the original Locality Preserving Projections (LPP), Unsupervised Discriminant Projection (UDP), Orthogonal LPP (OLPP), and other spectral mapping methods, the weight between two points is usually defined as a heat kernel or simply 0–1 weight, which cannot effectively reflect the sample class information. In Orthogonal Discriminant Projection (ODP), the weight between two points was defined based on their local information and class information, but it is not a monotonically decreasing with the increase of the distance between two nodes, so it is not very sound. In this paper, we first analyze the defect of the weight in ODP, then propose a novel weight measure between two nodes of a graph by combining their label information and local information, finally present a modified ODP algorithm following the ODP technique. The modified ODP algorithm can explore the intrinsic structure of original data and enhance the classification ability. The experimental results show that the modified ODP algorithm is effective and feasible. [Copyright &y& Elsevier]

Abstract: Extreme learning machine (ELM) is a new learning algorithm for single-hidden layer feedforward networks (SLFNs) proposed by Huang et al. . Its main advantage is the lower computational cost, which is especially relevant when dealing with many patterns defined in a high-dimensional space. This paper proposes an algorithm for pruning ELM networks by using regularized regression methods, thus obtaining a suitable number of the hidden nodes in the network architecture. Beginning from an initial large number of hidden nodes, irrelevant nodes are then pruned using ridge regression, elastic net and lasso methods; hence, the architectural design of ELM network can be automated. Empirical studies on several commonly used regression benchmark problems show that the proposed approach leads to compact networks that generate competitive results compared with the ELM algorithm. [Copyright &y& Elsevier]

Abstract: Selecting correct dimensions is very important to subspace clustering and is a challenging issue. This paper studies semi-supervised approach to the problem. In this setting, limited domain knowledge in the form of space level pair-wise constraints, i.e., must-links and cannot-links, are available. We propose a semi-supervised subspace clustering (S3C) algorithm that exploits constraint inconsistence for dimension selection. Our algorithm firstly correlates globally inconsistent constraints to dimensions in which they are consistent, then unites constraints with common correlating dimensions, and finally forms the subspaces according to the constraint unions. Experimental results show that S3C is superior to the typical unsupervised subspace clustering algorithm FINDIT, and the other constraint based semi-supervised subspace clustering algorithm SC-MINER. [Copyright &y& Elsevier]

Abstract: The objective of this paper is to present an efficient hardware architecture for generalized Hebbian algorithm (GHA). In the architecture, the principal component computation and weight vector updating of the GHA are operated in parallel, so that the throughput of the circuit can be significantly enhanced. In addition, the weight vector updating process is separated into a number of stages for lowering area costs and increasing computational speed. To show the effectiveness of the circuit, a texture classification system based on the proposed architecture is designed. It is embedded in a system-on-programmable-chip (SOPC) platform for physical performance measurement. Experimental results show that the proposed architecture is an efficient design for attaining both high speed performance and low area costs. [Copyright &y& Elsevier]

Abstract: Hybrid strategy, which generalizes a specific single-label algorithm while one or two data decomposition tricks are applied implicitly or explicitly, has become an effective and efficient tool to design and implement various multi-label classification algorithms. In this paper, we extend traditional binary support vector machine by introducing an approximate ranking loss as its empirical loss term to build a novel support vector machine for multi-label classification, resulting into a quadratic programming problem with different upper bounds of variables to characterize label correlation of individual instance. Further, our optimization problem can be solved via combining one-versus-rest data decomposition trick with modified binary support vector machine, which dramatically reduces computational cost. Experimental study on ten multi-label data sets illustrates that our method is a powerful candidate for multi-label classification, compared with four state-of-the-art multi-label classification approaches. [Copyright &y& Elsevier]

Abstract: Low-rank representations have received a lot of interest in the application of kernel-based methods. However, these methods made an assumption that the spectrum of the Gaussian or polynomial kernels decays rapidly. This is not always true and its violation may result in performance degradation. In this paper, we propose an effective technique for learning low-rank Mercer kernels (LMK) with fast-decaying spectrum. What distinguishes our kernels from other classical kernels (Gaussian and polynomial kernels) is that the proposed always yields low-rank Gram matrices whose spectrum decays rapidly, no matter what distribution the data are. Furthermore, the LMK can control the decay rate. Thus, our kernels can prevent performance degradation while using the low-rank approximations. Our algorithm has favorable in scalability—it is linear in the number of data points and quadratic in the rank of the Gram matrix. Empirical results demonstrate that the proposed method learns fast-decaying spectrum and significantly improves the performance. [Copyright &y& Elsevier]

Abstract: Although kernel minimum squared error (KMSE) is computationally simple, i.e., it only needs solving a linear equation set, it suffers from the drawback that in the testing phase the computational efficiency decreases seriously as the training samples increase. The underlying reason is that the solution of Naïve KMSE is represented by all the training samples in the feature space. Hence, in this paper, a method of selecting significant nodes for KMSE is proposed. During each calculation round, the presented algorithm prunes the training sample making least contribution to the objective function, hence called as PLOC-KMSE. To accelerate the training procedure, a batch of so-called nonsignificant nodes is pruned instead of one by one in PLOC-KMSE, and this speedup algorithm is named MPLOC-KMSE for short. To show the efficacy and feasibility of the proposed PLOC-KMSE and MPLOC-KMSE, the experiments on benchmark data sets and real-world instances are reported. The experimental results demonstrate that PLOC-KMSE and MPLOC-KMSE require the fewest significant nodes compared with other algorithms. That is to say, their computational efficiency in the testing phase is best, thus suitable for environments having a strict demand of computational efficiency. In addition, from the performed experiments, it is easily known that the proposed MPLOC-KMSE accelerates the training procedure without sacrificing the computational efficiency of testing phase to reach the almost same generalization performance. Finally, although PLOC and MPLOC are proposed in regression domain, they can be easily extended to classification problem and other algorithms such as kernel ridge regression. [Copyright &y& Elsevier]

Abstract: To address two problems, namely nonlinear problem and singularity problem, of linear discriminant analysis (LDA) approach in face recognition, this paper proposes a novel kernel machine-based rank-lifting regularized discriminant analysis (KRLRDA) method. A rank-lifting theorem is first proven using linear algebraic theory. Combining the rank-lifting strategy with three-to-one regularization technique, the complete regularized methodology is developed on the within-class scatter matrix. The proposed regularized scheme not only adjusts the projection directions but tunes their corresponding weights as well. Moreover, it is shown that the final regularized within-class scatter matrix approaches to the original one as the regularized parameter tends to zero. Two public available databases, namely FERET and CMU PIE face databases, are selected for evaluations. Compared with some existing kernel-based LDA methods, the proposed KRLRDA approach gives superior performance. [Copyright &y& Elsevier]

Abstract: Recently, the gradient (subgradient) projection method, especially by incorporating the idea of Nesterov''s method, has aroused more and more attention and achieved great successes on constrained optimization problems arising in the field of machine learning, data mining and signal processing. In the gradient projection method, a critical step is how to efficiently project a vector onto a constraint set. In this paper, we propose a unified method called Piecewise Root Finding (PRF) to efficiently calculate Euclidean projections onto three typical constraint sets: , Elastic Net (EN) and the Intersection of a Hyperplane and a Halfspace (IHH). In our PRF method, we first formulate a Euclidean projection problem as a root finding problem. Then, a Piecewise Root Finding algorithm is applied to find the root and global convergence is guaranteed. Finally, the Euclidean projection result is obtained as a function of the found root in a closed form. Moreover, the sparsity of the projected vector is considered, leading to reduced computational cost for projection onto the and EN. Empirical studies demonstrate that our PRF algorithm is efficient by comparing it with several state of the art algorithms for Euclidean projections onto the three typical constraint sets mentioned above. Besides, we apply our efficient Euclidean projection algorithm (PRF) to the Gradient Projection with Nesterov''s Method (GPNM), which efficiently solves the popular logistic regression problem with the /EN/IHH constraint. Experimental results on real-world data sets indicate that GPNM has a fast convergence speed. [Copyright &y& Elsevier]

Abstract: In this paper a new learning algorithm is proposed for the problem of simultaneous learning of a function and its derivatives as an extension of the study of error minimized extreme learning machine for single hidden layer feedforward neural networks. Our formulation leads to solving a system of linear equations and its solution is obtained by Moore–Penrose generalized pseudo-inverse. In this approach the number of hidden nodes is automatically determined by repeatedly adding new hidden nodes to the network either one by one or group by group and updating the output weights incrementally in an efficient manner until the network output error is less than the given expected learning accuracy. For the verification of the efficiency of the proposed method a number of interesting examples are considered and the results obtained with the proposed method are compared with that of other two popular methods. It is observed that the proposed method is fast and produces similar or better generalization performance on the test data. [Copyright &y& Elsevier]

Abstract: Image deblurring is a basic and important task of image processing. Traditional filtering based image deblurring methods, e.g. enhancement filters, partial differential equation (PDE) and etc., are limited by the hypothesis that natural images and noise are with low and high frequency terms, respectively. Noise removal and edge protection are always the dilemma for traditional models. In this paper, we study image deblurring problem from a brand new perspective—classification. And we also generalize the traditional PDE model to a more general case, using the theories of calculus of variations. Furthermore, inspired by the theories of approximation of functions, we transform the operator-learning problem into a coefficient-learning problem by means of selecting a group of basis, and build a filter-learning model. Based on extreme learning machine (ELM) , an algorithm is designed and a group of filters are learned effectively. Then a generalized image deblurring model, learned filtering PDE (LF-PDE), is built. The experiments verify the effectiveness of our models and the corresponding learned filters. It is shown that our model can overcome many drawbacks of the traditional models and achieve much better results. [Copyright &y& Elsevier]

Abstract: A robust training algorithm for a class of single-hidden layer feedforward neural networks (SLFNs) with linear nodes and an input tapped-delay-line memory is developed in this paper. It is seen that, in order to remove the effects of the input disturbances and reduce both the structural and empirical risks of the SLFN, the input weights of the SLFN are assigned such that the hidden layer of the SLFN performs as a pre-processor, and the output weights are then trained to minimize the weighted sum of the output error squares as well as the weighted sum of the output weight squares. The performance of an SLFN-based signal classifier trained with the proposed robust algorithm is studied in the simulation section to show the effectiveness and efficiency of the new scheme. [Copyright &y& Elsevier]

Abstract: The paper presents an indirect adaptive neural control scheme for a general high-order nonlinear continuous system. In the proposed scheme a neural controller is constructed based on the single-hidden layer feedforward network (SLFN) for approximating the unknown nonlinearities of dynamic systems. A sliding mode controller is also incorporated to compensate for the modelling errors of SLFN. The parameters of the SLFN are modified using the recently proposed neural algorithm named extreme learning machine (ELM), where the parameters of the hidden nodes are assigned randomly. However different from the original ELM algorithm, the output weights are updated based on the Lyapunov synthesis approach to guarantee the stability of the overall control system, even in the presence of modelling errors which are offset using the sliding mode controller. Finally the proposed adaptive neural controller is applied to control the inverted pendulum system with two different reference trajectories. The simulation results demonstrate that good tracking performance is achieved by the proposed control scheme. [Copyright &y& Elsevier]

Abstract: Variational Bayes learning or mean field approximation is widely used in statistical models which are made of mixtures of exponential distributions, for example, normal mixtures, binomial mixtures, and hidden Markov models. To derive variational Bayes learning algorithm, we need to determine the hyperparameters in the a priori distribution; however, the design method of hyperparameters has not yet been established. In the present paper, we propose two different design methods of hyperparameters which are applied to the different purposes. In the former method, the hyperparameter is determined for minimization of the generalization error. In the latter method, it is chosen so that candidates of hidden structure in training data are extracted. It is experimentally shown that the optimal hyperparameters for two purposes are different from each other. [Copyright &y& Elsevier]

Abstract: During the early stages of language acquisition, young infants face the task of learning a basic vocabulary without the aid of prior linguistic knowledge. Attempts have been made to model this complex behaviour computationally, using a variety of machine learning algorithms, a.o. non-negative matrix factorization (NMF). In this paper, we replace NMF in a vocabulary learning setting with a conceptually similar algorithm, probabilistic latent semantic analysis (PLSA), which can learn word representations incrementally by Bayesian updating. We further show that this learning framework is capable of modelling certain cognitive behaviours, e.g. forgetting, in a simple way. [Copyright &y& Elsevier]

Abstract: Recently, a new fast learning algorithm called Extreme Learning Machine (ELM) has been developed for Single-Hidden Layer Feedforward Networks (SLFNs) in G.-B. Huang, Q.-Y. Zhu and C.-K. Siew “[Extreme learning machine: theory and applications,” Neurocomputing 70 (2006) 489–501]. And, ELM has been successfully applied to many classification and regression problems. In this paper, the ELM algorithm is further studied for ordinal regression problems (named ORELM). We firstly proposed an encoding-based framework for ordinal regression which includes three encoding schemes: single multi-output classifier, multiple binary-classifications with one-against-all (OAA) decomposition method and one-against-one (OAO) method. Then, the SLFN was redesigned for ordinal regression problems based on the proposed framework and the algorithms are trained by the extreme learning machine in which input weights are assigned randomly and output weights can be decided analytically. Lastly widely experiments on three kinds of datasets were carried to test the proposed algorithm. The comparative results with such traditional methods as Gaussian Process for Ordinal Regression (ORGP) and Support Vector for Ordinal Regression (ORSVM) show that ORELM can obtain extremely rapid training speed and good generalization ability. Especially when the data set’s scalability increases, the advantage of ORELM will become more apparent. Additionally, ORELM has the following advantages, including the capabilities of learning in both online and batch modes and handling non-linear data. [ABSTRACT FROM AUTHOR]

Abstract: Graph construction plays a key role on learning algorithms based on graph Laplacian. However, the traditional graph construction approaches of ε-neighborhood and k-nearest-neighbor need to predefine the same neighbor parameter ε (or k) for all samples, which usually suffers from the difficulty of parameter selection and generally fail to effectively fit intrinsic structures of data. To mitigate these limitations to a certain extent, in this paper we present a novel and sample-dependent approach of graph construction, and name the so-constructed graph as Sample-dependent Graph (SG). Specifically, instead of predefining the same neighbor parameter for all samples, the SG depends on samples in question to determine neighbors of each sample and similarities between sample pairs. As a result, it not only avoids the intractability and high expense of neighbor parameter selection but also can more effectively fit the intrinsic structures of data. Further, in order to show the effectiveness of the SG, we apply it to the dimensionality reduction based on graph embedding, and incorporate it into the state-of-the-art off-the-shelf unsupervised locality preserving projection (LPP) to develop the sample-dependent LPP (SLPP). SLPP naturally inherits the merits of SG and maintains the attractive properties of the traditional LPP. The experiments on the toy and benchmark (UCI, face recognition, object category and handwritten digits recognition) datasets show the effectiveness and feasibility of the SG and SLPP with promising results. [ABSTRACT FROM AUTHOR]

Abstract: Any realistic model of learning from samples must address the issue of noisy data. AdaBoost is known as an effective method for improving the performance of base classifiers both theoretically and empirically. However, previous studies have shown that AdaBoost is prone to overfitting, especially in noisy domains. On the other hand, the kNN rule is one of the oldest and simplest methods for pattern classification. Nevertheless, it often yields competitive results, and in certain domains, when cleverly combined with prior knowledge, it has significantly advanced the state-of-the-art. In this paper, an edited AdaBoost by weighted kNN (EAdaBoost ) is designed where AdaBoost and kNN naturally complement each other. First, AdaBoost is run on the training data to capitalize on some statistical regularity in the data. Then, a weighted kNN algorithm is run on the feature space composed of classifiers produced by AdaBoost to achieve competitive results. AdaBoost is then used to enhance the classification accuracy and avoid overfitting by editing the data sets using the weighted kNN algorithm for improving the quality of training data. Experiments performed on ten different UCI data sets show that the new Boosting algorithm almost always achieves considerably better classification accuracy than AdaBoost. Furthermore, experiments on data with artificially controlled noise indicate that the new Boosting algorithm is robust to noise. [ABSTRACT FROM AUTHOR]

Abstract: Real Adaboost ensembles with weighted emphasis (RA-we) on erroneous and critical (near the classification boundary) samples have recently been proposed, leading to improved performance when an adequate combination of these terms is selected. However, finding the optimal emphasis adjustment is not an easy task. In this paper, we propose to make a fusion of the outputs of RA-we ensembles trained with different emphasis adjustments by means of a generalized voting scheme. The resulting committee of RA-we ensembles can retain the performance of the best RA-we component and even, occasionally, can improve it. Additionally, we present an ensemble selection strategy that removes from the committee RA-we ensembles with very poor performance. Experimental results show that these committees frequently outperform RA and RA-we with cross validated emphasis. [Copyright &y& Elsevier]

Abstract: Graph embedding is a general framework for subspace learning. However, because of the well-known outlier-sensitiveness disadvantage of the L2-norm, conventional graph embedding is not robust to outliers which occur in many practical applications. In this paper, an improved graph embedding algorithm (termed LPP-L1) is proposed by replacing L2-norm with L1-norm. In addition to its robustness property, LPP-L1 avoids small sample size problem. Experimental results on both synthetic and real-world data demonstrate these advantages. [Copyright &y& Elsevier]

Abstract: The problem of scheduling nonpreemtable tasks on parallel identical machines under constraint on discrete resource and requiring, additionally, renewable continuous resource to minimize the schedule length is considered in the paper. A continuous resource is divisible continuously and is allocated to tasks from given intervals in amounts unknown in advance. Task processing rate depends on the allocated amount of the continuous resource. To eliminate time-consuming optimal continuous resource allocation, a problem Θ Z with continuous resource discretisation is introduced. Because Θ Z is NP-hard a population-learning algorithm (PLA2) is proposed to tackle the problem. PLA2 is a population-based approach which takes advantage of the features common to the social education system rather than to the evolutionary processes. The proposed approach is based on the idea of constructing the hybrid algorithm integrating different optimization techniques complementing each other and producing a synergetic effect. Experimental results proved that PLA2 excels known algorithms for solving the considered problem. [Copyright &y& Elsevier]

Abstract: To conduct binary classification with highly imbalanced data is a very common problem, especially when the examples of interest are relatively rare. In this paper, we proposed the “Meta Imbalanced Classification Ensemble (MICE)” algorithm in order to dilute the effect of imbalanced data. In the MICE, the majority group is partitioned based on the transformed features from “inner product” to retain the geometric relation between two groups. The empirical results show that the performance of MICE is better than some renowned classification methods in terms of the specificity and the sensitivity. [Copyright &y& Elsevier]

Abstract: In this paper, a robust wavelet-based adaptive neural control (RWANC) with a PI type learning algorithm is proposed. The proposed RWANC system is composed of a wavelet neural controller and a fuzzy compensation controller. The wavelet neural control is utilized to approximate an ideal controller and the fuzzy compensation controller with a fuzzy logic system in it is used to remove the chattering phenomena of conventional sliding-mode control completely. In the RWANC, the learning algorithm is derived based on the Lyapunov function, thus the closed-loop system''s stability can be guaranteed. The chaotic system control has become an emerging topic in engineering community since the uncontrolled system displays complex, noisy-like and unpredictable behavior. Therefore, the proposed RWANC approach is applied to a second-order chaotic nonlinear system to investigate the effectiveness. Through the simulation results, the proposed RWANC scheme can achieve favorable tracking performance and the convergence of the tracking error and control parameters can be accelerated by the developed PI adaptation learning algorithm. [Copyright &y& Elsevier]

Abstract: Building recognition is a relatively specific recognition task in object recognition, which is challenging since it encounters rotation, scaling, illumination changes, occlusion, etc. Subspace learning, which dominates dimensionality reduction, has been widely exploited in computer vision research in recent years. It consists of classical linear dimensionality reduction methods, manifold learning, etc. To explore how different subspace learning algorithms affect building recognition, some representative algorithms, i.e., principal component analysis, linear discriminant analysis, locality preserving projections (unsupervised/supervised), and semi-supervised discriminant analysis, are applied for dimensionality reduction. Moreover, a building recognition scheme based on biologically-inspired feature extraction is proposed in this paper. Experiments undertaken on our own building database demonstrate that the proposed scheme embedded with subspace learning can achieve satisfactory results. [Copyright &y& Elsevier]

Abstract: Feature selection plays an important role in pattern classification. Its purpose is to remove redundant features from data set as many as possible. The presence of useless features may not only deteriorate the performance of learning algorithms, but also obscure important information (e.g., intrinsic structure) behind data. Along with new and emerging techniques, data sets in many domains are becoming larger and larger and many irrelevant features are often prevailing in these data sets. This, however, poses great challenges to traditional learning algorithms, such as low efficiency and over-fitting. Thus, it becomes apparent that an efficient technique is needed to eliminate redundant or irrelevant features from the data sets. Currently, many endeavors to cope with this problem have been attempted and various outstanding feature selection methods have been proposed. Unlike other selection methods, in this paper we propose a general scheme of boosting feature selection method using information metric. The primary characteristic of our method is that it exploits weight of data to select salient features. Furthermore, the weight of data will be dynamically changed after each candidate feature has been selected. Thus, the information criteria used in feature selector can exactly represent the relevant degree between features and the class labels. As a result, the selected feature subset has maximal relevance to the class labels. Simulation studies carried out on UCI data sets show that the classification performance achieved by our proposed method is better than those of other selection methods in most cases. [Copyright &y& Elsevier]

Abstract: In this paper, we propose VQ methods based on ensemble learning algorithms Bagging and AdaBoost. The proposed methods consist of more than one weak learner, which are trained in parallel or sequentially. In Bagging, the weak learners are trained in parallel by using randomly selected data from a given data set. The output for Bagging is given as the average among the weak learners. In AdaBoost, the weak learners are sequentially trained. The first weak learner is trained by using randomly selected data from a given data set. For the second and later weak learners, the probability distribution of learning data is modified so that each weak learner focuses on data involving higher error for the previous weak one. The output for AdaBoost is given as the weighted average among the weak learners. The presented simulation results show that the proposed methods can achieve a good performance in shorter learning times than conventional ones such as K-means and NG. [Copyright &y& Elsevier]

Abstract: The major drawbacks of backpropagation algorithm are local minima and slow convergence. This paper presents an efficient technique ANMBP for training single hidden layer neural network to improve convergence speed and to escape from local minima. The algorithm is based on modified backpropagation algorithm in neighborhood based neural network by replacing fixed learning parameters with adaptive learning parameters. The developed learning algorithm is applied to several problems. In all the problems, the proposed algorithm outperform well. [Copyright &y& Elsevier]

Abstract: In this paper, we present a fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks (FAOS-PFNN), where a novel structure learning algorithm incorporating a pruning strategy into new growth criteria is developed. The proposed growing procedure without pruning not only speeds up the online learning process but also facilitates a more parsimonious fuzzy neural network while achieving comparable performance and accuracy by virtue of the growing and pruning strategy. The FAOS-PFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In the parameter learning phase, all the free parameters of hidden units, regardless of whether they are newly created or originally existing, are updated by the extended Kalman filter (EKF) method. The effectiveness and superiority of the FAOS-PFNN paradigm is compared with other popular approaches like resource allocation network (RAN), RAN via the extended Kalman filter (RANEKF), minimal resource allocation network (MRAN), adaptive-network-based fuzzy inference system (ANFIS), orthogonal least squares (OLS), RBF-AFS, dynamic fuzzy neural networks (DFNN), generalized DFNN (GDFNN), generalized GAP-RBF (GGAP-RBF), online sequential extreme learning machine (OS-ELM) and self-organizing fuzzy neural network (SOFNN) on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world regression problems. Simulation results demonstrate that the proposed FAOS-PFNN algorithm can achieve faster learning speed and more compact network structure with comparably high accuracy of approximation and generalization. [Copyright &y& Elsevier]

Abstract: In a fully complex-valued feed-forward network, the convergence of the Complex-valued Back Propagation (CBP) learning algorithm depends on the choice of the activation function, learning sample distribution, minimization criterion, initial weights and the learning rate. The minimization criteria used in the existing versions of CBP learning algorithm in the literature do not approximate the phase of complex-valued output well in function approximation problems. The phase of a complex-valued output is critical in telecommunication and reconstruction and source localization problems in medical imaging applications. In this paper, the issues related to the convergence of complex-valued neural networks are clearly enumerated using a systematic sensitivity study on existing complex-valued neural networks. In addition, we also compare the performance of different types of split complex-valued neural networks. From the observations in the sensitivity analysis, we propose a new CBP learning algorithm with logarithmic performance index for a complex-valued neural network with exponential activation function. The proposed CBP learning algorithm directly minimizes both the magnitude and phase errors and also provides better convergence characteristics. Performance of the proposed scheme is evaluated using two synthetic complex-valued function approximation problems, the complex XOR problem, and a non-minimum phase equalization problem. Also, a comparative analysis on the convergence of the existing fully complex and split complex networks is presented. [Copyright &y& Elsevier]