Search

This paper researches the state estimation problem for a class of T-S fuzzy system which subject to unknown input. For this set of system, two approaches are studied to design the UIF. A converted system is constructed in the first method to solve the insolubility problem of inequalities because of the existence of the unknown input. The second method establishes filter structure and achieves the purpose of estimating the system states through adopting the direct way. By employing unknown input filter, the paper makes a smooth estimation not only to the system states, but also to the estimate signals and unknown disturbance inputs. At last, a simulation example is used to illustrate the effectiveness of the proposed UIF. [ABSTRACT FROM AUTHOR]

This paper aims to solve the keypoint correspondence problem in lunar surface images, a typical correspondence task under point ambiguity. Point ambiguity may be caused by repetitive patterns, cluttered scenes, and outliers in the images, which makes the local descriptors less discriminative. In this paper we introduce locally affine-invariance constraints on graphs to tackle the keypoint correspondence problem under point ambiguity. The key idea is that each point can be represented with the affine combination of its neighbors. It is suitable for our problem because it is not only invariant to scale and rotational change, but also more resistant to outliers. Specifically, we introduce the locally affine-invariance constraints into the subgraph matching problem and the common subgraph matching problem. The locally affine-invariance constraint is not directly applicable on common subgraph matching due to its dependency on awareness of selected keypoints. This problem is approximately addressed by solving a series of reliable matching identification and rematching problems. In the experiments, we first apply the proposed method on standard graph matching datasets to evaluate its effectiveness on general correspondence problem under point ambiguity, and second validate the applicability on the lunar surface image dataset. [ABSTRACT FROM AUTHOR]

Visual recognition of crowd dynamics has had a huge impact on several applications including surveillance, situation awareness, homeland security and intelligent environments. However, the state-of-the-art in crowd analysis has become diverse due to factors such as: (a) the underlying definition of a crowd, (b) the constituent elements of the crowd processing model, (c) its application, hence (d) the dataset and (e) the evaluation criteria available for benchmarking. Although such diversity is healthy, the techniques for crowd modelling thus developed have failed to establish credibility therefore becoming unreliable and of questionable validity across different research disciplines. This paper aims to give an account of such issues by deducing key statistical evidence from the existing literature and providing recommendations towards focusing on the general aspects of techniques rather than any specific algorithm. The advances and trends in crowd analysis are drawn in the context of crowd modelling studies published in leading journals and conferences over the past 7 years. Finally, this paper shall also provide a qualitative and quantitative comparison of some existing methods using various publicly available crowd datasets to substantiate some of the theoretical claims. [ABSTRACT FROM AUTHOR]

In design optimization problems, the robustness of a solution is essential to be considered when it is going to be implemented or used in real world. Robustness can be defined in different ways, but in this work, it is considered as the property of a solution’s performance to be as insensitive as possible to perturbations in that solution. Optimizing both quality and robustness is generally addressed as a multiobjective problem. This paper presents a complete framework aided by decision maker, which can be used to solve robust optimization problems with one or more objectives. Decision maker actively takes part of the process, defining a priori the robustness metric, and specifying iteratively a robustness factor that joins quality and robustness metrics into a single fitness function. The framework is very flexible allowing decision maker not only to specify the right robustness level for him, but also to exchange metrics and algorithms if he wants. The framework was tested and validated using two single-objective robust benchmarking functions, and in one new multiobjective robust function, created to challenge the framework, which tries to seek the most robust Pareto optimal front according to decision maker preferences. [ABSTRACT FROM AUTHOR]

Linear Discriminant Analysis (LDA) is a standard tool for classification and dimension reduction in many applications. However, the problem of high dimension is still a great challenge for the classical LDA. In this paper we consider the supervised pattern classification in the high dimensional setting, in which the number of features is much larger than the number of observations and present a novel approach to the sparse optimal scoring problem using the zero-norm. The difficulty in treating the zero-norm is overcome by using appropriate continuous approximations such that the resulting problems are solved by alternating schemes based on DC (Difference of Convex functions) programming and DCA (DC Algorithms). The experimental results on both simulated and real datasets show the efficiency of the proposed algorithms compared to the five state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Nonlinear optimal control (NOC) problem with complex dynamic constraints (CDC) is difficult to compute even with direct method. In this paper, a hybrid two-stage approach integrating an improved backtracking search optimization algorithm (IBSA) with the hp-adaptive Gauss pseudo-spectral methods (hpGPM) is proposed. Firstly, BSA is improved to enhance its convergent speed and the global search ability, by adopting the harmony search strategy and an adaptive amplitude control factor with individual optimum fitness feedback. Then, at the beginning stage of the hybrid search process, an initialization generator is constructed using IBSA to find a near optimum solution. When the change in fitness function approaches to a predefined value which is small enough, the search process is replaced by hpGPM to accelerate the search process and find an accurate solution. By this way, the hybrid algorithm is able to find a global optimum more quickly and accurately. Two NOC problems with CDC are examined using the proposed algorithm, and the corresponding Monte Carlo simulations are conducted. The comparison results show the hybrid algorithm achieves better performance in convergent speed, accuracy and robustness. [ABSTRACT FROM AUTHOR]

This paper is discussed with the problem of passivity based synchronization for a class of complex dynamical networks (CDNs) consisting uncertain inner coupling matrix together with successive time-varying delays via a state feedback delayed controller. Due to occurrence of uncertainties in coupling strengths, the considered CDNs take account of an uncertain inner coupling strength which is more general than the previously existing inner coupling strengths. Specifically, the uncertainties encountering in coupling terms are characterized with the aid of interval matrix approach. Also, by introducing a simple linear transformation, the corresponding error system is formulated. Then, based on the information about control delay term, two cases are considered namely, differentiable and non-differentiable. More precisely, by constructing an appropriate Lyapunov–Krasovskii functional (LKF) containing triple integral terms in respect of Kronecker product, for both the cases, some sufficient criteria are established in terms of linear matrix inequalities (LMIs) to guarantee the robust synchronization of the addressed CDNs based on passivity property. And the established criteria optimistically reduce the L 2 gain level from the disturbance to the output vector. Subsequently, the desired state feedback gain matrix is designed in terms of the solution to a convex optimization problem. Finally, a numerical example is presented to demonstrate the effectiveness of the proposed theoretical results. [ABSTRACT FROM AUTHOR]

Minimum component analysis is an elegant and promising approach to system identification. This paper focuses on developing related recursive algorithms. Two algorithms are presented. In the first approach, rank-one modification is used to get an accurate and quickly convergent algorithm. However, its drawback is low computational efficiency. Therefore, the second algorithm is proposed to realize faster computation. The simulation results verify the effectiveness of both algorithms. [ABSTRACT FROM AUTHOR]

This paper introduces an effective approach to study the stability of static neural networks with interval time-varying delay using delay partitioning approach and tighter integral inequality lemma. By decomposing the delay interval into multiple equidistant subintervals and multiple nonuniform subintervals, some suitable Lyapunov–Krasovskii functionals are constructed on these intervals. A set of novel sufficient conditions are obtained to guarantee the stability analysis issue for the considered system. These conditions are expressed in the framework of linear matrix inequalities, which heavily depend on the lower and upper bounds of the time-varying delay. It is shown, by comparing with existing approaches, that the delay-partitioning approach can largely reduce the conservatism of the stability results. Finally, three examples are given to show the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

This paper studies the problem of adaptive neural output feedback controller design for a class of uncertain switched stochastic nonlinear systems in strict-feedback form. In the design procedure, a common coordinate transformation for all subsystems is constructed to overcome the design difficulty caused by adoption of different coordinate transformation for different subsystems. Then, by using switched state observer to estimate the unmeasured states and different update laws for different subsystems, a novel neural output-feedback controller is designed via backstepping approach. Furthermore, based on the Lyapunov method and the average dwell time condition, the stability of the resulting closed-loop system can be achieved. It is shown that all the signals of the closed-loop system are bounded under a class of switching signals with average dwell time. Finally, simulation results are included for validating the advantages of the proposed approaches. [ABSTRACT FROM AUTHOR]

This paper analyzes stochastic stability and performance of discrete-time Cubature Kalman Filtering (CKF). The main contribution is (1) Boundedness analysis based on the constructor is proposed. Through Taylor expansion and simplifications of the non-linear transfer function, the constructor expression based on the variance and the structure error can be got. Through boundedness analysis on various subitems of the constructor expression, it is proved that when the initial error and variance are small enough, the estimation error remains bounded. (2) It is shown that the performance difference between CKF (Cubature Kalman Filtering) and Unscented Kalman Filtering (UKF) is the capture ability of the high items in the Taylor expansion. The performance is also related to the dimensions. Simulations are conducted to verify the theoretical analysis. [ABSTRACT FROM AUTHOR]

This paper is concerned with the problems of finite-time filter analysis and design for discrete time-delay neural networks with Markovian jump parameters and sector-bounded activation functions represented by Takagi–Sugeno fuzzy model. Firstly, a sufficient admissibility criterion is presented to guarantee that the augmented fuzzy jump neural network without parameter uncertainties is stochastically finite-time bounded in a prescribed time interval. From the admissibility criterion obtained, a sufficient condition on stochastic finite-time boundedness is provided for the augmented fuzzy jump neural networks. Then, a sufficient criterion to design a finite-time fuzzy filter is presented with uncertain parameters and Markovian jumps for discrete time-delay fuzzy neural networks. Moreover, conditions on stochastic finite-time stability are also established for nominal and uncertain delayed fuzzy jump neural networks without the presence of external disturbance. All criteria obtained can be represented as the form of linear matrix inequalities. Finally, numerical examples are given to illustrate the effectiveness of the obtained results. [ABSTRACT FROM AUTHOR]

This paper proposes a novel region-based active contour model in the level set formulation for medical image segmentation. We define a unified fitting energy framework based on Gaussian probability distributions to obtain the maximum a posteriori probability (MAP) estimation. The energy term consists of a global energy term to characterize the fitting of global Gaussian distribution according to the intensities inside and outside the evolving curve, as well as a local energy term to characterize the fitting of local Gaussian distribution based on the local intensity information. In the resulting contour evolution that minimizes the associated energy, the global energy term accelerates the evolution of the evolving curve far away from the objects, while the local energy term guides the evolving curve near the objects to stop on the boundaries. In addition, a weighting function between the local energy term and the global energy term is proposed by using the local and global variances information, which enables the model to select the weights adaptively in segmenting images with intensity inhomogeneity. Extensive experiments on both synthetic and real medical images are provided to evaluate our method, show significant improvements on both efficiency and accuracy, as compared with the popular methods. [ABSTRACT FROM AUTHOR]

In this paper, we combine decentralized adaptive control with the fractional-order techniques to investigate the synchronization of fractional-order complex-variable dynamical networks. A new lemma is proposed for estimating the Caputo fractional derivatives of Hermitian quadrtic Lyapunov functions. Based on local information among neighboring nodes, an effective fractional-order decentralized adaptive strategy to tune the coupling gains among network nodes is designed. This analysis is further extended to the case where only a small fraction of coupling gains are choosen to be adjusted. By constructing suitable Lyapunov functions and utilizing the proposed lemma, two sufficient criteria are derived to guarantee the network synchronization by using the proposed adaptive laws. Finally, numerical examples are given to validate the theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, a general framework named Filippov-framework and some new analytic techniques are presented for dealing with the exponential synchronization of memristor-based neural network system with time-varying delay. Several new sufficient conditions are established to guarantee two different types of exponential synchronization of the coupled networks based on master–slave (drive-response) concept and discontinuous state-feedback controller. In addition, we also provide an estimation of the exponential synchronization rate which depends on the time delay and system parameters. These conditions are good improvement and extension of the existing works on synchronization control of memristive or switching networks. Finally, the validity of our method and theoretical results is demonstrated by concrete examples with numerical simulations. [ABSTRACT FROM AUTHOR]

This paper presents an autopilot design for a robotic unmanned surface vehicle in the presence of unknown yaw dynamics and measurement noises. A robust adaptive steering law is developed with the aid of a predictor, neural networks, and a modified dynamic surface control technique. Specifically, a predictor together with a low-frequency learning-based neural updating law is developed to identify the unknown yaw dynamics, as well as to reconstruct the states corrupted by measurement noises. Besides, to avoid the noise amplification effect of first-order filter, a linear tracking differentiator is incorporated into the dynamic surface control design approach to produce a noise-tolerant estimate of virtual control derivative. The stability of the closed-loop autopilot system is established by Lyapunov analysis. A salient feature of the developed controller is that it can achieve the desired performance in the presence of model uncertainty and measurement noises, simultaneously. Both simulation and experiment results are given to validate the efficacy of the proposed method. [ABSTRACT FROM AUTHOR]

This paper addresses the novel architectures׳ formulation and linear matrix inequality (LMI)-based design of dynamic nonlinear anti-windup compensator (AWC) for nonlinear time-delay systems with continuous interval time-varying delays under input saturation. An internal model control (IMC)-based AWC architecture is suggested for stable nonlinear time-delay systems and, in addition, a decoupling AWC architecture, applicable to a broader class of nonlinear time-delay systems, is proposed for compensation of the undesirable saturation effects. Further, a correspondent decoupled architecture is derived and recommended for characterizing the delayed nonlinear AWC synthesis goals. By employing Lyapunov–Krasovskii functional, local sector condition, Lipschitz condition, L 2 gain minimization, and the delay-interval information, several sufficient conditions are derived for the design of nonlinear time-delay AWC. Numerical examples for FitzHugh–Nagumo neuron and Hopfield neural network under input saturation and time-delay are presented to reveal effectiveness of the proposed anti-windup approach. [ABSTRACT FROM AUTHOR]

This paper provides an undulatory locomotion model of C. elegans to achieve the chemotaxis behaviors based on the biological neuronal and neuromuscular structure. The on-cell and off-cell mechanism, as well as the proprioceptive mechanism is incorporated into the locomotion model. The nervous system of C. elegans is modeled by a dynamic neural network (DNN) that involves two parts: head DNN and motor neurons. The head DNN perceives the outside concentrations and generates the undulatory wave to the body. The motor neurons are responsible for transiting the undulatory wave along the body. The body of C. elegans is represented as a multi-joint rigid link model with 11 links. The undulatory locomotion behavior is achieved by using the DNN to control the lengths of muscles on ventral and dorsal sides, and then using the muscle lengths to control the angles between two consecutive links. In this work, the relations between the outputs of DNN and muscle lengths, as well as the muscle lengths and the angles between two consecutive links, are determined. Furthermore, owing to the learning capability of DNN, a set of nonlinear functions that are designed to represent the chemotaxis behaviors of C. elegans are learned by the head DNN. The testing results show good performance of the locomotion model for the chemotaxis behaviors of finding food and avoiding toxin, as well as slight and Ω turns. At last, quantitative analyses by comparing with the experiment results are provided to verify the realness and effectiveness of the locomotion model, which could serve as a prototype for other footless animals. [ABSTRACT FROM AUTHOR]

Dynamic model is an important issue for research on stability, dynamic characteristics, surge and control technique of axial-flow pumping system, and such a model is usually characterized by complex nonlinearity, strong coupling and time-varying mathematical equation. For the convenience of establishing model and highly effective computing, dynamic characteristics of the whole system are divided into four parts: pump lift-flow characteristics, pipeline characteristics, mechanical characteristics of asynchronous motor and torque characteristics of pump load. Each part is a nonlinear subsystem, and there are complex coupling relations among each other. In the paper, each part of the pump system is modeled respectively by mechanisms of hydrodynamics, transmission dynamics, electromechanics and affinity law. Considering that the axial-flow pump is characterized by nonlinearity and parameters are difficultly estimated in the low flow operation area and that the data of pump head-flow can be easily tested under the speed of power frequency, a modeling method combined with the RBF neural network is proposed, where hidden layer parameters are optimized by K means clustering algorithm, and the weights are trained by least square method. At last the whole simulation model of the axial-flow pumping system is set up, and the validity of the proposed modeling method is verified through simulation. [ABSTRACT FROM AUTHOR]

A unified associative memory model with a novel method for designing associative memories is presented in this paper. Based on continuous recurrent neural networks, bipolar patterns inputted from external can cause the output of neural networks to be memorized patterns. In the method, two conditions relevant to external inputs are derived to ensure the network states converge to a stable interval, and an exponential stable criterion is proposed for the network being a bipolar associative memory with higher recall speed. By introducing a tunable slope activation function and considering time delay, the proposed model is general and can recall the memorized patterns in auto-associative and hetero-associative way, while higher robust and more flexible memory can be obtained through the proposed method. Experimental verification demonstrates the effectiveness and generalization of the proposed method. [ABSTRACT FROM AUTHOR]

The success of SVM in solving pattern recognition problems has encouraged researcher to extend the development of different versions. They are well-known for their robustness and good generalization performance. In many real-world applications, the data to be trained are available on-line in a sequential fashion and because of space and time requirements, batch training methods are not suitable. This paper proposes a new fast on-line algorithm called OTWISVM. It defines two optimization problems and incremental learning is done based of them. Two hyperplanes are generated as decision functions thus each of them is closer to one of the two classes and is as far as possible from the other. The solution is constructed via two subsets of linearly independent samples seen so far, and is always bounded. Good accuracy and notable speed of the method was tested and affirmed both on ordinary and noisy data sets as opposed to similar algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes an improved stochastic second order learning algorithm for supervised neural network training. The proposed algorithm, named bounded stochastic diagonal Levenberg–Marquardt (B-SDLM), utilizes both gradient and curvature information to achieve fast convergence while requiring only minimal computational overhead than the stochastic gradient descent (SGD) method. B-SDLM has only a single hyperparameter as opposed to most other learning algorithms that suffer from the hyperparameter overfitting problem due to having more hyperparameters to be tuned. Experiments using the multilayer perceptron (MLP) and convolutional neural network (CNN) models have shown that B-SDLM outperforms other learning algorithms with regard to the classification accuracies and computational efficiency (about 5.3% faster than SGD on the mnist-rot-bg-img database). It can classify all testing samples correctly on the face recognition case study based on AR Purdue database. In addition, experiments on handwritten digit classification case studies show that significant improvements of 19.6% on MNIST database and 17.5% on mnist-rot-bg-img database can be achieved in terms of the testing misclassification error rates (MCRs). The computationally expensive Hessian calculations are kept to a minimum by using just 0.05% of the training samples in its estimation or updating the learning rates once per two training epochs, while maintaining or even achieving lower testing MCRs. It is also shown that B-SDLM works well in the mini-batch learning mode, and we are able to achieve 3.32 × performance speedup when deploying the proposed algorithm in a distributed learning environment with a quad-core processor. [ABSTRACT FROM AUTHOR]

In this paper, the fault-tolerant control (FTC) problem is investigated for a class of uncertain switched stochastic nonlinear systems. The controlled system contains unknown nonlinear functions and actuator faults, which are modeled as both lock-in-place and loss of effectiveness. With the help of fuzzy logic systems to approximate the unknown nonlinear functions and by utilizing the common Lyapunov function (CLF) method, a fuzzy adaptive state-feedback fault-tolerant tracking control approach is developed. It is shown that all the variables of the closed-loop system are semi-globally uniformly bounded (SGUUB) in probability, the tracking error converges to an arbitrary small neighborhood of the origin. A simulation is given to demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

In this paper, we consider the quantized distributed optimization problem over general directed digital multi-agent networks, where the communication channels have limited data transmission rates. To solve the optimization problem, a distributed quantized subgradient algorithm is presented among agents. Based on an encoder–decoder scheme and a zoom-in technique, we can achieve not only a consensus, but also an optimal solution. In particular, we study two cases of the quantization levels of each connected directed digital communication channel. One is under the case that the quantization levels are time-varying at each time step, and the other is under the case of fixed quantization level. Two rigorous theoretical analyses are performed and the optimal solutions can be obtained asymptotically. Moreover, the upper bound of the quantization levels at each time step and the convergence rate are analytically characterized. The effectiveness of proposed algorithm is demonstrated by two illustrative examples. [ABSTRACT FROM AUTHOR]

Abstract: Recently, Gandomi and Alavi proposed a meta-heuristic optimization algorithm, called Krill Herd (KH), for global optimization [Gandomi AH, Alavi AH. Krill Herd: A New Bio-Inspired Optimization Algorithm. Communications in Nonlinear Science and Numerical Simulation, 17(12), 4831–4845, 2012.]. This paper represents an optimization method to global optimization using a novel variant of KH. This method is called the Stud Krill Herd (SKH). Similar to genetic reproduction mechanisms added to KH method, an updated genetic reproduction schemes, called stud selection and crossover (SSC) operator, is introduced into the KH during the krill updating process dealing with numerical optimization problems. The introduced SSC operator is originated from original Stud genetic algorithm. In SSC operator, the best krill, the Stud, provides its optimal information for all the other individuals in the population using general genetic operators instead of stochastic selection. This approach appears to be well capable of solving various functions. Several problems are used to test the SKH method. In addition, the influence of the different crossover types on convergence and performance is carefully studied. Experimental results indicate an instructive addition to the portfolio of swarm intelligence techniques. [Copyright &y& Elsevier]

Abstract: Hybrid strategy, which generalizes a specific single-label algorithm while one or two data decomposition tricks are applied implicitly or explicitly, has become an effective and efficient tool to design and implement various multi-label classification algorithms. In this paper, we extend traditional binary support vector machine by introducing an approximate ranking loss as its empirical loss term to build a novel support vector machine for multi-label classification, resulting into a quadratic programming problem with different upper bounds of variables to characterize label correlation of individual instance. Further, our optimization problem can be solved via combining one-versus-rest data decomposition trick with modified binary support vector machine, which dramatically reduces computational cost. Experimental study on ten multi-label data sets illustrates that our method is a powerful candidate for multi-label classification, compared with four state-of-the-art multi-label classification approaches. [Copyright &y& Elsevier]

Abstract: In this paper, one novel genetic algorithm dynamic chain-like agent genetic algorithm (CAGA) is proposed for solving global numerical optimization problem and feature selection problem. The CAGA combines the chain-like agent structure with dynamic neighboring genetic operators to get higher optimization capability. An agent in chain-like agent structure represents a candidate solution to the optimization problem. Any agent interacts with neighboring agents to evolve. With dynamic neighboring genetic operators, they compete and cooperate with their neighbors, and can use knowledge to increase energies. Global numerical optimization problem and feature selection problem are the most important problems for evolutionary algorithm, especially for genetic algorithm. Hence, the experiments of global numerical optimization and feature selection are necessary to verify the performance of genetic algorithms. Corresponding experiments have been done and show that CAGA is suitable for real coding and binary coding optimization problems, and has more precise and more stable optimization results. [Copyright &y& Elsevier]

Abstract: Isometric embedding approaches can nicely deal with noiseless data sets, but they are topologically unstable when confronted with sparse data sets or with data sets containing a large amount of noise and outliers, as where the neighborhood is critically distorted. Inspired from the cognitive relativity, this paper proposes a relative transformation that can be applied to build the relative space from the original space of data. In relative space, the noise and outliers will become further away from the normal points, while the near points will become relative closer. Accordingly we determine the neighborhood in the relative space for isometric embedding, while the embedding is still performed in the original space. The conducted experiments on both synthetic and real data sets validate the approach. [Copyright &y& Elsevier]

Abstract: The problem addressed in this paper concerns the prototype reduction for a nearest-neighbor classifier. An efficient method based on particle swarm optimization is proposed here for finding a good set of prototypes. Starting from an initial random selection of a small number of training patterns, we generate a set of prototypes, using the particle swarm optimization, which minimizes the error rate on the training set. To improve the classification performance, during the training phase the prototype generation is repeated N times, then each of the resulting N sets of prototypes is used to classify each test pattern, and finally these N classification results are combined by the “vote rule”. The performance improvement with respect to the state-of-the-art approaches is validated through experiments with several benchmark datasets. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a novel robust support vector machine based on the smooth Ramp loss, which has strong ability of suppressing the influences of outliers. The concave–convex procedure (CCCP) is utilized to transform the associated non-convex optimization into a convex one. Then, a Newton-type algorithm is developed to solve the resulting primal optimization of robust support vector machine, and the convergence property and the complexity are discussed. Experimental results show that the proposed approach has significant robustness to outliers and yields better generalization performance than the classical support vector machines on both synthetic and real data sets. [Copyright &y& Elsevier]

Abstract: The eng-genes concept involves the use of fundamental known system functions as activation functions in a neural model to create a ‘grey-box’ neural network. One of the main issues in eng-genes modelling is to produce a parsimonious model given a model construction criterion. The challenges are that (1) the eng-genes model in most cases is a heterogenous network consisting of more than one type of nonlinear basis functions, and each basis function may have different set of parameters to be optimised; (2) the number of hidden nodes has to be chosen based on a model selection criterion. This is a mixed integer hard problem and this paper investigates the use of a forward selection algorithm to optimise both the network structure and the parameters of the system-derived activation functions. Results are included from case studies performed on a simulated continuously stirred tank reactor process, and using actual data from a pH neutralisation plant. The resulting eng-genes networks demonstrate superior simulation performance and transparency over a range of network sizes when compared to conventional neural models. [Copyright &y& Elsevier]

Abstract: This paper presents an improved particle swarm optimization (PSO) and discrete PSO (DPSO) with an enhancement operation by using a self-adaptive evolution strategies (ES). This improved PSO/DPSO is proposed for joint optimization of three-layer feedforward artificial neural network (ANN) structure and parameters (weights and bias), which is named ESPNet. The experimental results on two real-world problems show that ESPNet can produce compact ANNs with good generalization ability. [Copyright &y& Elsevier]

In this paper we present a method of parameter optimization, relative trust-region learning, where the trust-region method and the relative optimization [M. Zibulevsky, Blind source separation with relative Newton method, in: Proceedings of the ICA, Nara, Japan, 2003, pp. 897–902] are jointly exploited. The relative trust-region method finds a direction and a step size with the help of a quadratic model of the objective function (as in the conventional trust-region methods) and updates parameters in a multiplicative fashion (as in the relative optimization). We apply this relative trust-region learning method to the problem of independent component analysis (ICA), which leads to the relative TR-ICA algorithm which turns out to possess the equivariant property (as in the relative gradient) and to achieve faster convergence than the relative gradient and even Newton-type algorithms. Empirical comparisons with several existing ICA algorithms demonstrate the useful behavior of the relative TR-ICA algorithm, such as the equivariant property and fast convergence. [Copyright &y& Elsevier]

Abstract: This paper proposes a new method to obtain sparseness and structure detection for a class of kernel machines related to least-squares support vector machines (LS-SVMs). The key method is to adopt an hierarchical modeling strategy. Here, the first level consists of an LS-SVM substrate which is based upon an LS-SVM formulation with additive regularization trade-off. This regularization trade-off is determined at higher levels such that sparse representations and/or structure detection are obtained. Using the necessary and sufficient conditions for optimality given by the Karush–Kuhn–Tucker conditions, one can guide the interaction between different levels via a well-defined set of hyper-parameters. From a computational point of view, all levels can be fused into a single convex optimization problem. Furthermore, the principle is applied in order to optimize the validation performance of the resulting kernel machine. Sparse representations as well as structure detection are obtained, respectively, by using an regularization scheme and a measure of maximal variation at the second level. A number of case studies indicate the usefulness of these approaches both with respect to interpretability of the final model as well as generalization performance. [Copyright &y& Elsevier]