Search

For ensemble learning, how to select and combine the candidate classifiers are two key issues which influence the performance of the ensemble system dramatically. Random vector functional link networks (RVFL) without direct input-to-output links is one of suitable base-classifiers for ensemble systems because of its fast learning speed, simple structure and good generalization performance. In this paper, to obtain a more compact ensemble system with improved convergence performance, an improved ensemble of RVFL based on attractive and repulsive particle swarm optimization (ARPSO) with double optimization strategy is proposed. In the proposed method, ARPSO is applied to select and combine the candidate RVFL. As for using ARPSO to select the optimal base RVFL, ARPSO considers both the convergence accuracy on the validation data and the diversity of the candidate ensemble system to build the RVFL ensembles. In the process of combining RVFL, the ensemble weights corresponding to the base RVFL are initialized by the minimum norm least-square method and then further optimized by ARPSO. Finally, a few redundant RVFL is pruned, and thus the more compact ensemble of RVFL is obtained. Moreover, in this paper, theoretical analysis and justification on how to prune the base classifiers on classification problem is presented, and a simple and practically feasible strategy for pruning redundant base classifiers on both classification and regression problems is proposed. Since the double optimization is performed on the basis of the single optimization, the ensemble of RVFL built by the proposed method outperforms that built by some single optimization methods. Experiment results on function approximation and classification problems verify that the proposed method could improve its convergence accuracy as well as reduce the complexity of the ensemble system. [ABSTRACT FROM AUTHOR]

In the Artificial Bee Colony (ABC) algorithm, the employed bee and the onlooker bee phase involve updating the candidate solutions by changing a value in one dimension, dubbed one-dimension update process. For some problems which the number of dimensions is very high, the one-dimension update process can cause the solution quality and convergence speed drop. This paper proposes a new algorithm, using reinforcement learning for solution updating in ABC algorithm, called R-ABC. After updating a solution by an employed bee, the new solution results in positive or negative reinforcement applied to the solution dimensions in the onlooker bee phase. Positive reinforcement is given when the candidate solution from the employed bee phase provides a better fitness value. The more often a dimension provides a better fitness value when changed, the higher the value of update becomes in the onlooker bee phase. Conversely, negative reinforcement is given when the candidate solution does not provide a better fitness value. The performance of the proposed algorithm is assessed on eight basic numerical benchmark functions in four categories with 100, 500, 700, and 900 dimensions, seven CEC2005’s shifted functions with 100, 500, 700, and 900 dimensions, and six CEC2014’s hybrid functions with 100 dimensions. The results show that the proposed algorithm provides solutions which are significantly better than all other algorithms for all tested dimensions on basic benchmark functions. The number of solutions provided by the R-ABC algorithm which are significantly better than those of other algorithms increases when the number of dimensions increases on the CEC2005’s shifted functions. The R-ABC algorithm is at least comparable to the state-of-the-art ABC variants on the CEC2014’s hybrid functions. [ABSTRACT FROM AUTHOR]

Conjugate gradient methods are widely used for solving unconstrained optimization and nonlinear equations, specially in large-scale cases. Since they own the attractive practical factors of simple computation and low memory requirement, interesting theoretical features of curvature information and strong global convergence. In this paper, we present a modified conjugate gradient algorithm by line search method with acceleration scheme for nonlinear symmetric equations. Furthermore, the proposed method not only possess descent property but also owns global convergence in mild conditions. Numerical results also indicate that the presented method is much more effective than the other methods for the test problems. [ABSTRACT FROM PUBLISHER]

In this paper, a multiple-time-step decoupled algorithm for a non-stationary Stokes–Darcy problem is proposed and investigated. Under a modest time step restriction of physical parameters and the time step proportion, we give the stability analysis and convergence analysis of the decoupled scheme with different time step in fluid and porous subregions. Finally, a series of numerical experiments are provided to illustrate the accuracy, efficiency, and stability of the presented method for the coupled problem with the Beavers–Joseph–Saffman–Jones interface conditions. [ABSTRACT FROM PUBLISHER]

This paper presents a stable and fast algorithm for independent component analysis with reference (ICA-R). This is a technique for incorporating available reference signals into the ICA contrast function so as to form an augmented Lagrangian function under the framework of constrained ICA (cICA). The previous ICA-R algorithm was constructed by solving the optimization problem via a Newton-like learning style. Unfortunately, the slow convergence and potential misconvergence limit the capability of ICA-R. This paper first investigates and probes the flaws of the previous algorithm and then introduces a new stable algorithm with a faster convergence speed. There are two other highlights in this paper: first, new approaches, including the reference deflation technique and a direct way of obtaining references, are introduced to facilitate the application of ICA-R; second, a new method is proposed that the new ICA-R is used to recover the complete underlying sources with new advantages compared with other classical ICA methods. Finally, the experiments on both synthetic and real-world data verify the better performance of the new algorithm over both previous ICA-R and other well-known methods. [ABSTRACT FROM AUTHOR]

We introduce a new iterative algorithm for finding a common element of a fixed point problem of amenable semigroups of nonexpansive mappings, the set solutions of a system of a general system of generalized equilibria in a real Hilbert space. Then, we prove the strong convergence of the proposed iterative algorithm to a common element of the above three sets under some suitable conditions. As applications, at the end of the paper, we apply our results to find the minimum-norm solutions which solve some quadratic minimization problems. The results obtained in this paper extend and improve many recent ones announced by many others. [ABSTRACT FROM AUTHOR]

In this paper, the Hager and Zhang (HZ) conjugate gradient (CG) method and the modified HZ (MHZ) CG method are presented for large-scale nonsmooth convex minimization. Under some mild conditions, convergent results of the proposed methods are established. Numerical results show that the presented methods can be better efficiency for large-scale nonsmooth problems, and several problems are tested (with the maximum dimensions to 100,000 variables). [ABSTRACT FROM AUTHOR]

Particle swarm optimization (PSO) is an evolutionary computing method based on intelligent collective behavior of some animals. It is easy to implement and there are few parameters to adjust. The performance of PSO algorithm depends greatly on the appropriate parameter selection strategies for fine tuning its parameters. Inertia weight (IW) is one of PSO’s parameters used to bring about a balance between the exploration and exploitation characteristics of PSO. This paper proposes a new nonlinear strategy for selecting inertia weight which is named Flexible Exponential Inertia Weight (FEIW) strategy because according to each problem we can construct an increasing or decreasing inertia weight strategy with suitable parameters selection. The efficacy and efficiency of PSO algorithm with FEIW strategy (FEPSO) is validated on a suite of benchmark problems with different dimensions. Also FEIW is compared with best time-varying, adaptive, constant and random inertia weights. Experimental results and statistical analysis prove that FEIW improves the search performance in terms of solution quality as well as convergence rate. [ABSTRACT FROM AUTHOR]

This paper considers the noncooperative maximization of mutual information in the vector Gaussian interference channel in a fully distributed fashion via game theory. This problem has been widely studied in a number of works during the past decade for frequency-selective channels, and recently for the more general multiple-input multiple-output (MIMO) case, for which the state-of-the art results are valid only for nonsingular square channel matrices. Surprisingly, these results do not hold true when the channel matrices are rectangular and/or rank deficient matrices. The goal of this paper is to provide a complete characterization of the MIMO game for arbitrary channel matrices, in terms of conditions guaranteeing both the uniqueness of the Nash equilibrium and the convergence of asynchronous distributed iterative waterfilling algorithms. Our analysis hinges on new technical intermediate results, such as a new expression for the MIMO waterfilling projection valid (also) for singular matrices, a mean-value theorem for complex matrix-valued functions, and a general contraction theorem for the multiuser MIMO watefilling mapping valid for arbitrary channel matrices. The quite surprising result is that uniqueness/convergence conditions in the case of tall (possibly singular) channel matrices are more restrictive than those required in the case of (full rank) fat channel matrices. We also propose a modified game and algorithm with milder conditions for the uniqueness of the equilibrium and con- vergence, and virtually the same performance (in terms of Nash equilibria) of the original game. [ABSTRACT FROM AUTHOR]

In this paper, we consider a set-valued implicit quasi-variational inequality problem in real uniformly smooth Banach space and show its equivalence with a fixed-point problem. Using this equivalence and Banach contraction principle for set-valued mapping, we prove the existence of solution for the set-valued implicit quasi-variational inequality problem. Further, we propose and analyze an iterative algorithm for the set-valued implicit quasi-variational inequality problem and discuss its convergence analysis and stability. The theorems presented in this paper generalize, improve and unify the results given in Noor [20,22] and the references therein. [ABSTRACT FROM AUTHOR]

This paper characterizes the existence and form of the possible limit error signals in typical parameter-optimal iterative learning control. The set of limit errors has attracting and repelling components and the behaviour of the algorithm in the vicinity of these sets can be associated with the undesirable properties of apparent (but in fact temporary) convergence or permanent slow convergence properties in practice. The avoidance of these behaviours in practice is investigated using novel switching strategies. Deterministic strategies are analysed to prove the feasibility of the concept by proving that each of a number of such strategies is guaranteed to produce global convergence of errors to zero independent of the details of plant dynamics. For practical applications a random switching strategy is proposed to replace these approaches and shown, by example, to produce substantial potential improvements when compared with the non-switching case. The work described in this paper is covered by pending patent applications in the UK and elsewhere. [ABSTRACT FROM AUTHOR]

The Lloyd algorithm originated in the context of optimal quantization and represents a fixed point iteration for computing an optimal quantizer. Reducing average distortion at every step, it constructs a Voronoi partition of the domain and replaces each generator with the centroid of the corresponding Voronoi cell. Optimal quantization is obtained in the case of a centroidal Voronoi tessellation (CVT), which is a special Voronoi tessellation of a domain Ω ϵ ℝd having the property that the generators of the Voronoi diagram are also the centers of mass, with respect to a given density function ? ⩾ 0, of the corresponding Voronoi cells. The Lloyd iteration is currently the most popular and elegant algorithm for computing CVTs and optimal quantizers, but many questions remain about its convergence, especially in d-dimensional spaces (d > 1). In this paper, we prove that any limit point of the Lloyd iteration in any dimensional spaces is nondegenerate provided that Ω is a convex and bounded set and ? belongs to L¹(Ω) and is positive almost everywhere. This ensures that the fixed point map remains closed and hence the standard theory of descent methods guarantees weak global convergence of the Lloyd iteration to the set of nondegenerate fixed point quantizers. While previously only conjectured, the convergence properties of the Lloyd iteration are rigorously justified under such minimal regularity assumptions on the density functional. The results presented in this paper go beyond existing convergence theories for CVTs and optimal quantization related algorithms and should be of interest to both the mathematical and engineering communities. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents the new convergence analysis for the Newton-like method proposed by [Xinyuan Wu, A new continuation Newton-like method and its deformation, Appl. Math. Comput. 112 (2000) 75–78.] Compared with the original version of the convergence analysis, the restriction imposed on f″(x) is removed thoroughly. In order to guarantee the quadratic convergence of the Newton-like method, it is suffices to suppose that f′(x ∗)≠0 and f′(x) is local Lipschitz near x ∗, where x ∗ is a solution of nonlinear equation f(x)=0. Moreover, some comments are given with examples for the Newton-like method, in comparison with Newton’s method. It can be concluded that the new algorithm is more feasible, effective than Newton’s method. In particularly, when it happens that, the derivative of the function f(x) at an iterate is singular or almost singular, the Newton-like method is vast superior to the classical Newton method. The numerical results of the paper strongly support the conclusion. [Copyright &y& Elsevier]

This paper proposes several globally convergent geometric optimization algorithms on Riemannian manifolds, which extend some existing geometric optimization techniques. Since any set of smooth constraints in the Euclidean space Rn (corresponding to constrained optimization) and the Rn space itself (corresponding to unconstrained optimization) are both special Riemannian manifolds, and since these algorithms are developed on general Riemannian manifolds, the techniques discussed in this paper provide a uniform framework for constrained and unconstrained optimization problems. Unlike some earlier works, the new algorithms have less restrictions in both convergence results and in practice. For example, global minimization in the one-dimensional search is not required. All the algorithms addressed in this paper are globally convergent. For some special Riemannian manifold other than Rn, the newalgorithms are very efficient. Convergence rates are obtained. Applications are discussed. [ABSTRACT FROM AUTHOR]

The active appearance model (AAM) is a powerful tool for modeling images of deformable objects and has been successfully used in a variety of alignment, tracking, and recognition applications. AAM uses subspace-based deformable models to represent the images of a certain object class. In general, fitting such complicated models to previously unseen images using standard optimization techniques is a computationally complex task because the gradient matrix has to be numerically computed at every iteration. The critical feature of AAM is a fast convergence scheme which assumes that the gradient matrix is fixed Around the optimal coefficients for all images. Our work in this paper starts with the observation that such a fixed gradient matrix inevitably specializes to a certain region in the texture space, and the fixed gradient matrix is not a good estimate of the actual gradient as the target texture moves away from this region. Hence, we propose an adaptive AAM algorithm that linearly adapts the gradient matrix according to the composition of the target image's texture to obtain a better estimate for the actual gradient. We show that the adaptive AAM significantly outperforms the basic AAM, especially in images that are particularly challenging for the basic algorithm. In terms of speed and accuracy, the idea of a linearly adaptive gradient matrix presented in this paper provides an interesting compromise between a standard optimization technique that recomputes the gradient at every iteration and the fixed gradient matrix approach of the basic AAM. [ABSTRACT FROM AUTHOR]

This paper proposes a new algorithm for the unconstrained optimization problem, which is derived as an application of the control theory called Direct Gradient Descent Control. The static optimization problem is solved by using a dynamic controller in which the convergence speed can be greatly accelerated. The major idea is to consider the objective function F(x) and its time derivative dF(x(t))/dt as the evaluation criteria, and to apply the gradient descent method. It is verified by simulation that the proposed dynamic mathematical programming has a very great capability to converge to the optimal solution. It is also interesting to note that the method has to a certain extent the ability to find not only local optimal solutions, but also global optimal solutions. This paper improves the basic algorithm of dynamic mathematical programming so that it is suited to global optimization. The new algorithm generates movement toward the global optimal solution by utilizing two different trajectories with interaction (the chaotic trajectory and convergence trajectory). It is verified by simulation that the new algorithm functions as a global optimization procedure with a high success probability. © 2005 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 88(6): 20–27, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.20094 [ABSTRACT FROM AUTHOR]

In this paper, a new sequential penalty algorithm, based on the L[sub ∞] exact penalty function, is proposed for a general nonlinear constrained optimization problem. The algorithm has the following characteristics: it can start from an arbitrary initial point; the feasibility of the subproblem is guaranteed; the penalty parameter is adjusted automatically; global convergence without any regularity assumption is proved. The update formula of the penalty parameter is new. It is proved that the algorithm proposed in this paper behaves equivalently to the standard SQP method after sufficiently many iterations. Hence, the local convergence results of the standard SQP method can be applied to this algorithm. Preliminary numerical experiments show the efficiency and stability of the algorithm. [ABSTRACT FROM AUTHOR]

This paper contains a new convergence analysis for the Lewis and Torczon generalized pattern search (GPS) class of methods for unconstrained and linearly constrained optimization. This analysis is motivated by a desire to understand the successful behavior of the algorithm under hypotheses that are satisfied by many practical problems. Specifically, even if the objective function is discontinuous or extended-valued, the methods find a limit point with some minimizing properties. Simple examples show that the strength of the optimality conditions at a limit point depends not only on the algorithm, but also on the directions it uses and on the smoothness of the objective at the limit point in question. The contribution of this paper is to provide a simple convergence analysis that supplies detail about the relation of optimality conditions to objective smoothness properties and to the defining directions for the algorithm, and it gives previous results as corollaries. [ABSTRACT FROM AUTHOR]

This is a review paper of the role of Carleman estimates in the theory of multidimensional coefficient inverse problems since the first inception of this idea in 1981. [ABSTRACT FROM AUTHOR]

It is well known that the convergence of multivariate subdivision schemes with finite masks can be characterized via joint spectral radius. For nonnegative masks, we will present in this paper some computable simply sufficient conditions for the convergence, which will cover a substantially large class of schemes. [ABSTRACT FROM AUTHOR]

The Nelder-Mead algorithm, a longstanding direct search method for unconstrained optimization published in 1965, is designed to minimize a scalar-valued function f of n real variables using only function values, without any derivative information. Each Nelder-Mead iteration is associated with a nondegenerate simplex defined by n + 1 vertices and their function values; a typical iteration produces a new simplex by replacing the worst vertex by a new point. Despite the method's widespread use, theoretical results have been limited: for strictly convex objective functions of one variable with bounded level sets, the algorithm always converges to the minimizer; for such functions of two variables, the diameter of the simplex converges to zero but examples constructed by McKinnon show that the algorithm may converge to a nonminimizing point. This paper considers the restricted Nelder-Mead algorithm, a variant that does not allow expansion steps. In two dimensions we show that for any nondegenerate starting simplex and any twice-continuously differentiable function with positive definite Hessian and bounded level sets, the algorithm always converges to the minimizer. The proof is based on treating the method as a discrete dynamical system and relies on several techniques that are nonstandard in convergence proofs for unconstrained optimization. [ABSTRACT FROM AUTHOR]

The behavior of the multiplicative acceleration of Selmer's algorithm is widely unknown and no general result on convergence has been detected yet. Solely for its 2-dimensional, periodic expansions, there exist some results on convergence and approximation due to Fritz Schweiger. In this paper we show that periodic expansions of any dimension do in fact converge and that the coordinates of the limit points are rational functions of the largest eigenvalue of the periodicity matrix. [ABSTRACT FROM AUTHOR]

In this paper, a method for solving constrained convex optimization problems is introduced. The problem is cast equivalently as a parametric unconstrained one, the (single) parameter being the optimal value of the original problem. At each stage of the algorithm the parameter is updated, and the resulting subproblem is only approximately solved. A linear rate of convergence of the parameter sequence is established. Using an optimal gradient method due to Nesterov [Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543 547] to solve the arising subproblems, it is proved that the resulting gradient-based algorithm requires an overall of O(log(1/ε)/√ε) inner iterations to obtain an ε-optimal and feasible solution. An image deblurring problem is solved, demonstrating the capability of the algorithm to solve large-scale problems within reasonable accuracy. [ABSTRACT FROM AUTHOR]

Abstract: A nonparametric adaptive filtering approach is proposed in this paper. The algorithm is obtained by exploiting a time-varying step size in the traditional NLMS weight update equation. The step size is adjusted according to the square of a time-averaging estimate of the autocorrelation of a priori and a posteriori error. Therefore, the new algorithm has more effective sense proximity to the optimum solution independent of uncorrelated measurement noise. Moreover, this algorithm has fast convergence at the early stages of adaptation and small final misadjustment at steady-state process. It works reliably and is easy to implement since the update function is nonparametric. Furthermore, the experimental results in system identification applications are presented to illustrate the principle and efficiency of the proposed algorithm. [Copyright &y& Elsevier]

The matrix rank minimization problem has applications in many fields, such as system identification, optimal control, low-dimensional embedding, etc. As this problem is NP-hard in general, its convex relaxation, the nuclear norm minimization problem, is often solved instead. Recently, Ma, Goldfarb and Chen proposed a fixed-point continuation algorithm for solving the nuclear norm minimization problem ( Math. Program., doi:, ). By incorporating an approximate singular value decomposition technique in this algorithm, the solution to the matrix rank minimization problem is usually obtained. In this paper, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving affinely constrained matrix rank minimization problems are reported. [ABSTRACT FROM AUTHOR]

Nonlinear optimization algorithms could be divided into local exploitation methods such as Nelder-Mead (NM) algorithm and global exploration ones, such as differential evolution (DE). The former searches fast yet could be easily trapped by local optimum, whereas the latter possesses better convergence quality. This paper proposes hybrid differential evolution and NM algorithm with re-optimization, called as DE-NMR. At first a modified NM, called NMR is presented. It re-optimizes from the optimum point at the first time and thus being able to jump out of local optimum, exhibits better properties than NM. Then, NMR is combined with DE. To deal with equal constraints, adaptive penalty function method is adopted in DE-NMR, which relaxes equal constraints into unequal constrained functions with an adaptive relaxation parameter that varies with iteration. Benchmark optimization problems as well as engineering design problems are used to experiment the performance of DE-NMR, with the number of function evaluation times being employed as the main index of measuring convergence speed, and objective function values as the main index of optimum's quality. Non-parametric tests are employed in comparing results with other global optimization algorithms. Results illustrate the fast convergence speed of DE-NMR. [ABSTRACT FROM AUTHOR]

Linear Programming, LP, problems with finite optimal value have a zero duality gap and a primal–dual strictly complementary optimal solution pair. On the other hand, there exist Semidefinite Programming, SDP, problems which have a nonzero duality gap (different primal and dual optimal values; not both infinite). The duality gap is assured to be zero if a constraint qualification, e.g., Slater’s condition (strict feasibility) holds. Measures of strict feasibility, also called distance to infeasibility, have been used in complexity analysis, and, it is known that (near) loss of strict feasibility results in numerical difficulties. In addition, there exist SDP problems which have a zero duality gap but no strict complementary primal–dual optimal solution. We refer to these problems as hard instances of SDP. The assumption of strict complementarity is essential for asymptotic superlinear and quadratic rate convergence proofs. In this paper, we introduce a procedure for generating hard instances of SDP with a specified complementarity nullity (the dimension of the common nullspace of primal–dual optimal pairs). We then show, empirically, that the complementarity nullity correlates well with the observed local convergence rate and the number of iterations required to obtain optimal solutions to a specified accuracy, i.e., we show, even when Slater’s condition holds, that the loss of strict complementarity results in numerical difficulties. We include two new measures of hardness that correlate well with the complementarity nullity. [ABSTRACT FROM AUTHOR]

This paper proposes an efficient method to solve the optimal power flow problem in power systems using Particle Swarm Optimization (PSO). The objective of the proposed method is to find the steady-state operating point which minimizes the fuel cost, while maintaining an acceptable system performance in terms of limits on generator power, line flow, and voltage. Three different inertia weights, a constant inertia weight (CIW), a time-varying inertia weight (TVIW), and global-local best inertia weight (GLbestIW), are considered with the particle swarm optimization algorithm to analyze the impact of inertia weight on the performance of PSO algorithm. The PSO algorithm is simulated for each of the method individually. It is observed that the PSO algorithm with the proposed inertia weight yields better results, both in terms of optimal solution and faster convergence. The proposed method has been tested on the standard IEEE 30 bus test system to prove its efficacy. The algorithm is computationally faster, in terms of the number of load flows executed, and provides better results than other heuristic techniques. [ABSTRACT FROM AUTHOR]

In this paper a new hybrid conjugate gradient algorithm is proposed and analyzed. The parameter β k is computed as a convex combination of the Polak-Ribière-Polyak and the Dai-Yuan conjugate gradient algorithms, i.e. β=(1− θ k) β+ θ k β. The parameter θ k in the convex combination is computed in such a way that the conjugacy condition is satisfied, independently of the line search. The line search uses the standard Wolfe conditions. The algorithm generates descent directions and when the iterates jam the directions satisfy the sufficient descent condition. Numerical comparisons with conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that this hybrid computational scheme outperforms the known hybrid conjugate gradient algorithms. [ABSTRACT FROM AUTHOR]

Let C be a nonempty closed convex subset of a Banach space E with the dual E*, let T: C→ E* be a Lipschitz continuous mapping and let S: C→ C be a relatively nonexpansive mapping. In this paper, by employing the notion of generalized projection operator, we study the following variational inequality (for short, VI( T− f, C)): find x∈ C such that where f∈ E* is a given element. Utilizing the modified Ishikawa iteration and the modified Halpern iteration for relatively nonexpansive mappings, we propose two modified versions of J.L. Li’s (J. Math. Anal. Appl. 295:115–126, ) iterative algorithm for finding approximate solutions of VI( T− f, C). Moreover, it is proven that these iterative algorithms converge strongly to the same solution of VI( T− f, C), which is also a fixed point of S. [ABSTRACT FROM AUTHOR]

In this paper, we are interested in some convergent formulations for the unsymmetric collocation method or the so-called Kansa’s method. We review some newly developed theories on solvability and convergence. The rates of convergence of these variations of Kansa’s method are examined and verified in arbitrary–precision computations. Numerical examples confirm with the theories that the modified Kansa’s method converges faster than the interpolant to the solution; that is, exponential convergence for the multiquadric and Gaussian radial basis functions (RBFs). Some numerical algorithms are proposed for efficiency and accuracy in practical applications of Kansa’s method. In double–precision, even for very large RBF shape parameters, we show that the modified Kansa’s method, through a subspace selection using a greedy algorithm, can produce acceptable approximate solutions. A benchmark algorithm is used to verify the optimality of the selection process. [ABSTRACT FROM AUTHOR]

As power systems becoming more closely interconnected and are being deregulated in energy markets, distributed simulations among different dispatch centers are highly required for online full system analysis and control applications. In this paper a new algorithm for distributed transient stability simulation of interconnected power systems is presented. Based on a Jacobian-free Newton-GMRES(m) method, this algorithm requires only exchanges of states of boundary buses among different regions. Therefore, it has strong scalability in distributed computing environments built on heterogeneous computing resources. Moreover, several accelerating methods are developed to enhance its efficiency, including continuous preconditioning with adaptive preconditioners, predicting boundary conditions and multistep coordination. The standard IEEE 39-bus system and a real power system with 1165 buses were used as test systems. The test results show that these accelerating methods greatly enhance the convergence rate of the proposed algorithm and reduce communication costs remarkably, which proves the novel algorithm is feasible and can be adopted in wide area networks with high-latency. [ABSTRACT FROM AUTHOR]

In this paper we present a new class of memory gradient methods for unconstrained optimization problems and develop some useful global convergence properties under some mild conditions. In the new algorithms, trust region approach is used to guarantee the global convergence. Numerical results show that some memory gradient methods are stable and efficient in practical computation. In particular, some memory gradient methods can be reduced to the BB method in some special cases. [ABSTRACT FROM AUTHOR]

In this paper, motivated by the result of Aoyama et al., we first introduce a new iterative scheme for an accretive operator in a Banach space and obtain some strong convergence theorems in a Banach space under appropriate conditions on parameters. [ABSTRACT FROM AUTHOR]

In this paper, we consider algorithms for constructing n-terms approximations with nonnegative coefficients. The convergence theorem is proved for a “positive” analog of the Pure Greedy Algorithm. We establish a condition on the sequence of weakness coefficients which is sufficient for the convergence of the Positive Weak Greedy Algorithm. This condition is also necessary for the class of monotone sequences. [ABSTRACT FROM AUTHOR]

Direct search optimization algorithms are becoming an important alternative to well-established gradient based methods. Due to the fact that a single cost function evaluation may take a substantial amount of time, optimization can be a lengthy process. In order to shorten the run time one often resorts to parallel algorithms. Asynchronous algorithms are particularly efficient since they have no synchronization points. This paper is an attempt to establish a convergence theory for a class of such parallel direct search algorithms. The notion of a search direction generator (SDG) is introduced. An algorithmic framework for parallel distributed optimization methods based on SDGs is presented along with the corresponding convergence theory. The theory almost completely decouples the stepsize control from the sufficient descent requirement, which is necessary for the finite termination of the algorithm's inner loop. The proposed framework has several attributes considered very favourable in loosely coupled parallel systems (e.g. clusters of workstations), such as fault tolerance and scalability. The framework is illustrated by optimizing a set of test problems on a cluster of workstations. In all tested cases, a speedup was obtained that increased with the increasing number of workstations. Fault tolerance and scalability of the framework were also demonstrated by removing and adding workstations to the cluster while an optimization run was in progress. [ABSTRACT FROM AUTHOR]

Abstract: A family of algorithms for approximate solution of the bound-constrained minimization problem was introduced in [K.A. Ariyawansa, W.L. Tabor, A class of collinear scaling algorithms for bound-constrained optimization: Derivation and computational results, Technical Report 2003-1, Department of Mathematics, Washington State University, Pullman, WA, 2003, submitted for publication. Available at http://www.math.wsu.edu/math/TRS/2003-1.pdf]. These algorithms employ the standard barrier method, with the inner iteration based on trust region methods. Local models are conic functions rather than the usual quadratic functions, and are required to match first and second derivatives of the barrier function at the current iterate. The various members of the family are distinguished by the choice of a vector-valued parameter, which is the zero vector in the degenerate case that quadratic local models are used. This paper presents a convergence analysis of the family of algorithms presented in [K.A. Ariyawansa, W.L. Tabor, A class of collinear scaling algorithms for bound-constrained optimization: Derivation and computational results, Technical Report 2003-1, Department of Mathematics, Washington State University, Pullman, WA, 2003, submitted for publication. Available at http://www.math.wsu.edu/math/TRS/2003-1.pdf]. Specifically, convergence properties similar to those of barrier methods using quadratic local models are established. [Copyright &y& Elsevier]

Abstract: In this paper, we discuss some more generalized iterative schemes for solving the large sparse saddle point problems proposed by Ling and Hu [X.F. Ling, X.Z. Hu, On the iterative algorithm for large sparse saddle point problems, Appl. Math. Comput. 178 (2006) 372–379.], and apply the preconditioned GMRES to accelerate the convergence. Numerical experiments are reported that show the effectiveness of the proposed methods. [Copyright &y& Elsevier]

In this paper we propose and justify at the differential level an iteration method for solving the Stokes problem in a rectangular domain with the rate of convergence independent of the ratio of the sides. The numerical implementation of the algorithm is constructed for the cavity test problem and computational results are given. [ABSTRACT FROM AUTHOR]

In this paper, we introduce the doubling transformation, a structure-preserving transformation for symplectic pencils, and present its basic properties. Based on these properties, a unified convergence theory for the structure-preserving doubling algorithms for a class of Riccati-type matrix equations is established, using only elementary matrix theory. [ABSTRACT FROM AUTHOR]

In this paper, by using a so-called fixed steplength strategy, we propose a BFGS method without use of line searches for unconstrained optimization. Under mild conditions, we show the global and superlinear convergence of the proposed method even when the objective function is nonconvex. [ABSTRACT FROM AUTHOR]

Abstract: In this paper the integration formulas of Gaussian methods with positive coefficient for fuzzy integrations are discussed and then are followed by convergence theorem. The proposed algorithms are illustrated and compared with Newton Cot’s methods [T. Allahviranloo, Newton Cot’s methods for integration of fuzzy functions, Appl. Math. Comp., in press] by solving some numerical examples. [Copyright &y& Elsevier]

The notion of weak sharp minima is an important tool in the analysis of the perturbation behavior of certain classes of optimization problems as well as in the convergence analysis of algorithms designed to solve these problems. It has been studied extensively by several authors. This paper is the second of a series on this subject where the basic results on weak sharp minima in Part I are applied to a number of important problems in convex programming. In Part II we study applications to the linear regularity and bounded linear regularity of a finite collection of convex sets as well as global error bounds in convex programming. We obtain both new results and reproduce several existing results from a fresh perspective. [ABSTRACT FROM AUTHOR]

The celebrated simplicity of the EM algorithm is somewhat lost in its common use for generalized linear mixed models (GLMMs) because of its analytically intractable E-step. A natural and typical strategy in practice is to implement the E-step via Monte Carlo by drawing the unobserved random effects from their conditional distribution as specified by the E-step. In this paper, we show that further augmenting the missing data (e.g., the random effects) used by the M-step leads to a quite attractive and general slice sampler for implementing the Monte Carlo E-step. The slice sampler scheme is straightforward to implement, and it is neither restricted to the particular choice of the link function (e.g., probit) nor to the distribution of the random effects (e.g., normal). We apply this scheme to the standard EM algorithm as well as to an alternative EM algorithm which treats the variance-standardized random effects, rather than the random effects themselves, as missing data. The alternative EM algorithm does not only have faster convergence, but also leads to generalized linear model-like variance estimation, because it converts the random-effect standard deviations into linear regression parameters. Using the well-known salamander mating problem, we compare these two algorithms with each other, as well as with a variety of methods given in the literature in terms of the resulting point and interval estimates. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a noniterative identification procedure for neuro-fuzzy based Hammerstein model is presented. The proposed method not only avoids the inevitable restrictions on static nonlinear function encountered by using the polynomial approach, but also overcomes the problems of initialization and convergence of the model parameters, which are usually resorted to trial and error procedure in the existing iterative algorithms used for the identification of Hammerstein model. To construct the neuro-fuzzy based model, a clustering algorithm is presented to estimate the centers and widths of the model, and an analytical solution is developed to calculate the weights of the model in a noniterative manner. Examples are used to illustrate the applicability of the proposed method and a comparison with polynomial approach is made. [Copyright &y& Elsevier]

Abstract: The paper presents a new rectangle branch-and-reduce approach for solving nonconvex quadratic programming problems, in which a new lower approximate linear functions of the quadratic function over an n-rectangle is given to determine a lower bound of the global optimal value of the original problem over each rectangle, and a simple two-partition technique on rectangle is used, as well as the tactics on reducing and deleting subrectangles is used to accelerate the convergence of the proposed algorithm. The proposed algorithm is proved to be convergent and shown to be effective with numerical results. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a new inversion free variant of the basic fixed point iteration method for obtaining a maximal positive definite solution of the nonlinear matrix equation . It is more accurate than Zhan''s algorithm (J. Sci. Comput. 17 (1996) 1167) and has less number of operations than the algorithm of Guo and Lancaster (Math. Comput. 68 (1999) 1589). We derive convergence conditions of the iteration and existence conditions of a solution to the problem. Finally, we give some numerical results to illustrate the behavior of the considered algorithm. [Copyright &y& Elsevier]

In this paper, we present an algorithm for the CDT subproblem. This problem stems from computing a trust region step of an algorithm, which was first proposed by Celis, Dennis and Tapia for equality constrained optimization. Our algorithm considers general case of the CDT subproblem, convergence of the algorithm is proved. Numerical examples are also provided. [ABSTRACT FROM AUTHOR]

This paper deals with a discrete monotone iterative algorithm for solving a non-linear singularly perturbed parabolic reaction–diffusion problem. The monotone iterative method, which is based on a weighted average difference scheme, is applied to computing a non-linear difference scheme obtained after discretization of the continuous problem. This method solves only linear discrete systems at each iterative step of the iterative process. The uniform convergence of the method is established and numerical experiments are presented. [ABSTRACT FROM AUTHOR]

Many current Evolutionary Algorithms (EAs) suffer from a tendency to converge prematurely or stagnate without progress for complex problems. This may be due to the loss of or failure to discover certain valuable genetic material or the loss of the capability to discover new genetic material before convergence has limited the algorithm's ability to search widely. In this paper, the Hierarchical Fair Competition (HFC) model, including several variants, is proposed as a generic framework for sustainable evolutionary search by transforming the convergent nature of the current EA framework into a non-convergent search process. That is, the structure of HFC does not allow the convergence of the population to the vicinity of any set of optimal or locally optimal solutions. The sustainable search capability of HFC is achieved by ensuring a continuous supply and the incorporation of genetic material in a hierarchical manner, and by culturing and maintaining, but continually renewing, populations of individuals of intermediate fitness levels. HFC employs an assembly-line structure in which subpopulations are hierarchically organized into different fitness levels, reducing the selection pressure within each subpopulation while maintaining the global selection pressure to help ensure the exploitation of the good genetic material found. Three EAs based on the HFC principle are tested - two on the even-10-parity genetic programming benchmark problem and a real-world analog circuit synthesis problem, and another on the HIFF genetic algorithm (GA) benchmark problem. The significant gain in robustness, scalability and efficiency by HFC, with little additional computing effort, and its tolerance of small population sizes, demonstrates its effectiveness on these problems and shows promise of its potential for improving other existing EAs for difficult problems. A paradigm shift from that of most EAs is proposed: rather than trying to escape from local optima or delay convergence at a local optimum, HFC allows the emergence of new optima continually in a bottom-up manner, maintaining low local selection pressure at all fitness levels, while fostering exploitation of high-fitness individuals through promotion to higher levels. [ABSTRACT FROM AUTHOR]