Search

This paper considers a stochastic optimization problem over the fixed point sets of quasinonexpansive mappings on Riemannian manifolds. The problem enables us to consider Riemannian hierarchical optimization problems over complicated sets, such as the intersection of many closed convex sets, the set of all minimizers of a nonsmooth convex function, and the intersection of sublevel sets of nonsmooth convex functions. We focus on adaptive learning rate optimization algorithms, which adapt step-sizes (referred to as learning rates in the machine learning field) to find optimal solutions quickly. We then propose a Riemannian stochastic fixed point optimization algorithm, which combines fixed point approximation methods on Riemannian manifolds with the adaptive learning rate optimization algorithms. We also give convergence analyses of the proposed algorithm for nonsmooth convex and smooth nonconvex optimization. The analysis results indicate that, with small constant step-sizes, the proposed algorithm approximates a solution to the problem. Consideration of the case in which step-size sequences are diminishing demonstrates that the proposed algorithm solves the problem with a guaranteed convergence rate. This paper also provides numerical comparisons that demonstrate the effectiveness of the proposed algorithms with formulas based on the adaptive learning rate optimization algorithms, such as Adam and AMSGrad. [ABSTRACT FROM AUTHOR]

In Dai and Wen (Numer. Algor. 69, 337–341 2015), some improvements have been presented in the proof of Theorem 2 and Theorem 4 in Andrei (Numer. Algor. 47, 143–156 2008). However, due to incorrect inequalities used, the reasoning proof of Theorem 2.1 in Dai and Wen (Numer. Algor. 69, 337–341 2015) is incorrect. Moreover, the assumption on 0 < c1 ≤ 𝜃k < 1 of Theorem 2.1 cannot be deleted in Dai and Wen (Numer. Algor. 69, 337–341 2015). In this paper, the necessary corrections are made. Finally, another version of the proof process of Theorem 3.1 in (Numer. Algor. 69, 337–341 2015) is given. Throughout, we use the same notations and equation numbers as in Dai and Wen (Numer. Algor. 69, 337–341 2015), Andrei (Numer. Algor. 47, 143–156 2008). [ABSTRACT FROM AUTHOR]

This paper focuses on efficient computational approaches to compute approximate solutions of a linear inverse problem that is contaminated with mixed Poisson-Gaussian noise, and when there are additional outliers in the measured data. The Poisson-Gaussian noise leads to a weighted minimization problem, with solution-dependent weights. To address outliers, the standard least squares fit-to-data metric is replaced by the Talwar robust regression function. Convexity, regularization parameter selection schemes, and incorporation of non-negative constraints are investigated. A projected Newton algorithm is used to solve the resulting constrained optimization problem, and a preconditioner is proposed to accelerate conjugate gradient Hessian solves. Numerical experiments on problems from image deblurring illustrate the effectiveness of the methods. [ABSTRACT FROM AUTHOR]

In this paper, the authors present an s-dependent conjugate gradient method for unconstrained optimization problem and make two different kinds of estimations of upper bounds of βk with respect to βkFR which are called dependent ratio. The global convergence of s-dependent GFR conjugate gradient method using several step-size rules is obtained. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a new concept of approximate optimal stepsize for gradient method, use it to interpret the Barzilai-Borwein (BB) method, and present an efficient gradient method with approximate optimal stepsize for large unconstrained optimization. If the objective function f is not close to a quadratic on a line segment between the current iterate xk and the latest iterate xk−1, we construct a conic model to generate the approximate optimal stepsize for gradient method if the conic model is suitable to be used. Otherwise, we construct a new quadratic model or two other new approximation models to generate the approximate optimal stepsize for gradient method. We analyze the convergence of the proposed method under some suitable conditions. Numerical results show the proposed method is very promising. [ABSTRACT FROM AUTHOR]

The matrix completion problem is easy to state: let A be a given data matrix in which some entries are unknown. Then, it is needed to assign 'appropriate values' to these entries. A common way to solve this problem is to compute a rank- k matrix, B , that approximates A in a least squares sense. Then, the unknown entries in A attain the values of the corresponding entries in B . This raises the question of how to determine a suitable matrix rank. The method proposed in this paper attempts to answer this question. It builds a finite sequence of matrices $B_{k}, k = 1, 2, \dots $ , where B is a rank- k matrix that approximates A in a least squares sense. The computational effort is reduced by using B as starting point in the computation of B . The ability of B to serve as substitute for A is measured with two objective functions: a 'training' function that measures the distance between the known part of A and the corresponding part of B , and a 'probe' function that assesses the quality of the imputed entries. Watching the changes in these functions as k increases enables us to find an optimal matrix rank. Numerical experiments illustrate the usefulness of the proposed approach. [ABSTRACT FROM AUTHOR]

In this paper, we propose conjugate gradient path method for solving derivative-free unconstrained optimization. The iterative direction is obtained by constructing and solving quadratic interpolation model of the objective function with conjugate gradient methods. The global convergence and local superlinear convergence rate of the proposed algorithm are established under some reasonable conditions. Finally, the numerical results are reported to show the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Several image restoration applications require the solution of nonnegatively constrained minimization problems whose objective function is typically constituted by the sum of a data fit function and a regularization function. Newton projection methods are very attractive because of their fast convergence, but they need an efficient implementation to avoid time consuming iterations. In this paper we present NPTool, a set of Matlab functions implementing Newton projection methods for image denoising and deblurring applications. They are specifically thought for two different data fit functions, the Least Squares function and the Kullback-Leibler divergence, and two regularization functions, Tikhonov and Total Variation, giving the opportunity of solving a large variety of restoration problems. The package is easily extensible to other linear or nonlinear data fit and regularization functions. Some examples of its use are included in the package and shown in this paper. [ABSTRACT FROM AUTHOR]

In this paper, we present primal-dual interior-point methods for convex quadratic optimization based on a finite barrier, which has been investigated earlier for the case of linear optimization by Bai et al. (SIAM J Optim 13(3):766-782, ). By means of the feature of the finite kernel function, we study the complexity analysis of primal-dual interior-point methods based on the finite barrier and derive the iteration bounds that match the currently best known iteration bounds for large- and small-update methods, namely, $O(\sqrt{n}\log{n}\log{\frac{n}{\varepsilon}})$ and $O(\sqrt{n}\log{\frac{n}{\varepsilon}})$, respectively, which are as good as the linear optimization analogue. Numerical tests demonstrate the behavior of the algorithms with different parameters. [ABSTRACT FROM AUTHOR]

In this paper, a class of additive block triangular preconditioners are constructed for solving block two-by-two linear systems with symmetric positive (semi-)definite sub-matrices. Convergence analysis of the related splitting iteration method shows that it is almost unconditionally convergent and behaves problem independent with a convergence rate less than 0.5 under a practical parameter choice. Optimization of the preconditioned matrices, which have real and tight eigenvalue distributions, shows that it can result in an upper bound less than 2 for the condition number of the preconditioned matrices. Moreover, we also give a special consideration about the feasibility of the proposed preconditioner for solving more general problems with indefinite sub-matrices. Numerical experiments based on examples arising from complex symmetric linear systems and PDE-constrained optimization problems are presented to show the robustness and effectiveness of the proposed preconditioners compared with some other existing preconditioners. [ABSTRACT FROM AUTHOR]

In this paper, a method is developed for solving nonsmooth nonconvex minimization problems. This method extends the classical BFGS framework. First, we generalize the Wolfe conditions for locally Lipschitz functions and prove that this generalization is well defined. Then, a line search algorithm is presented to find a step length satisfying the generalized Wolfe conditions. Next, the Goldstein e-subgradient is approximated by an iterative method and a descent direction is computed using a positive definite matrix. This matrix is updated using the BFGS method. Finally, a minimization algorithm based on the BFGS method is described. The algorithm is implemented in MATLAB and numerical results using it are reported. [ABSTRACT FROM AUTHOR]

In this paper, we propose Linesearch methods for solving a bilevel split variational inequality problem (BSVIP) involving a strongly monotone mapping in the upper-level problem and pseudomonotone mappings in the lower-level one. A strongly convergent algorithm for such a BSVIP is proposed and analyzed. [ABSTRACT FROM AUTHOR]

Finding the solution of linear complementarity problem (LCP) is an important subject in scientific computing, engineering and economic applications. In the literature, many iterative methods have been proposed, especially, when the matrix of the problem is a real positive definite or an H− matrix. In this paper, we propose a new method using an alternating direction (AD) technique based on a modified symmetric successive overrelaxation (MSSOR) for LCP, assuming that the real matrix of the LCP is symmetric positive definite. This approach is an inner/outer iteration method, in which AD method is used as an outer iteration, and the MSSOR method as an inner one. We provide a detailed proof for the convergence of the proposed method. In addition, numerical results show that the proposed method is feasible and effective. [ABSTRACT FROM AUTHOR]

Multiobjective optimization has a large number of real-life applications. Under this motivation, in this paper, we present a new method for solving multiobjective optimization problems with both linear constraints and bound constraints on the variables. This method extends, to the multiobjective setting, the classical reduced gradient method for scalar-valued optimization. The proposed algorithm generates a feasible descent direction by solving an appropriate quadratic subproblem, without the use of any scalarization approaches. We prove that the sequence generated by the algorithm converges to Pareto-critical points of the problem. We also present some numerical results to show the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a preconditioned general modulus-based matrix splitting iteration method for solving modulus equations arising from linear complementarity problems. Its convergence theory is proved when the system matrix is an H+-matrix, from which some new convergence conditions can be derived for the (general) modulus-based matrix splitting iteration methods. Numerical results further show that the proposed methods are superior to the existing methods. [ABSTRACT FROM AUTHOR]

Chaos optimization algorithm is a recently developed method for global optimization based on chaos theory. It has many good features such as easy implementation, short execution time and robust mechanisms for escaping from local minima compared with existing stochastic searching algorithms. In the present paper, we propose a new chaos optimization algorithm (COA) approach called SLC (symmetric levelled chaos) based on new strategies including symmetrization and levelling: the proposed SLC method is, to our knowledge, the first chaos approach that can efficiently and successfully operates in higher-dimensional spaces. The proposed method is tested on a number of benchmark functions, and its performance comparisons are provided against previous COAs. The experiment results show that the proposed method has a marked improvement in performance over the classical COA approaches. Moreover, among all COA approaches, SLC is the only one to work efficiently in higher-dimensional spaces. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to introduce a general iterative method for finding a common element of the set of common fixed points of an infinite family of nonexpansive mappings and the set of split variational inclusion problem in the framework Hilbert spaces. Strong convergence theorem of the sequences generated by the purpose iterative scheme is obtained. In the last section, we present some computational examples to illustrate the assumptions of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Using the Multivariate Decomposition Method (MDM), we develop an efficient algorithm for approximating the ∞-variate integral for a class of functions f that are once differentiable with respect to each variable. MDM requires efficient algorithms for d-variate versions of the problem. Such algorithms are provided by Smolyak's construction which is based on efficient algorithms for the univariate integration Detailed analysis and development of (nearly) optimal quadratures for I( f) is the main contribution of the current paper. [ABSTRACT FROM AUTHOR]

Kernel functions play an important role in the complexity analysis of the interior point methods for linear optimization. In this paper, we present a primal-dual interior point method for linear optimization based on a new kernel function consisting of a trigonometric function in its barrier term. By simple analysis, we show that the feasible primal-dual interior point methods based on the new proposed kernel function enjoys $O\left (\sqrt {n}\left (\log {n}\right )^{2}\log \frac {n}{\epsilon }\right )$ worst case complexity result which improves the results obtained by El Ghami et al. (J Comput Appl Math 236:3613-3623, 2012) for the kernel functions with trigonometric barrier terms. [ABSTRACT FROM AUTHOR]

In this paper, we propose a model-hybrid approach for nonlinear optimization that employs both trust region method and quasi-Newton method, which can avoid possibly resolve the trust region subproblem if the trial step is not acceptable. In particular, unlike the traditional trust region methods, the new approach does not use a single approximate model from beginning to the end, but instead employs quadratic model or conic model at every iteration adaptively. We show that the new algorithm preserves the strong convergence properties of trust region methods. Numerical results are also presented. [ABSTRACT FROM AUTHOR]

In this paper, a new filled function method for finding a global minimizer of global optimization is proposed. The proposed filled function is continuously differentiable and only contains one parameter. It has no parameter sensitive terms. As a result, a general classical local optimization method can be used to find a better minimizer of the proposed filled function with easy parameter adjustment. Numerical experiments show that the proposed filled function method is effective. [ABSTRACT FROM AUTHOR]

The limited memory BFGS method (L-BFGS) is an adaptation of the BFGS method for large-scale unconstrained optimization. However, The L-BFGS method need not converge for nonconvex objective functions and it is inefficient on highly ill-conditioned problems. In this paper, we proposed a regularization strategy on the L-BFGS method, where the used regularization parameter may play a compensation role in some sense when the condition number of Hessian approximation tends to become ill-conditioned. Then we proposed a regularized L-BFGS method and established its global convergence even when the objective function is nonconvex. Numerical results show that the proposed method is efficient. [ABSTRACT FROM AUTHOR]

This paper presents a modified ODE-based algorithm for unconstrained optimization problems. It combines the idea of IMPBOT algorithm with nonmonotone and subspace techniques. The main feature of this method is that at each iteration, a lower dimensional system of linear equations is solved to obtain a trial step. Under some standard assumptions, the method is proven to be globally convergent. Numerical results show the efficiency of this proposed method in practical computation. [ABSTRACT FROM AUTHOR]

In this paper, we proposed an implementation of stochastic perturbation of reduced gradient and bisection (SPRGB) method for optimizing a non-convex differentiable function subject to linear equality constraints and non-negativity bounds on the variables. In particular, at each iteration, we compute a search direction by reduced gradient, and optimal line search by bisection algorithm along this direction yields a decrease in the objective value. SPRGB method is desired to establish the global convergence of the algorithm. An implementation and tests of SPRGB algorithm are given, and some numerical results of large-scale problems are presented, which show the efficient of this approach. [ABSTRACT FROM AUTHOR]

This paper proposes and analyzes an affine scaling trust-region method with line search filter technique for solving nonlinear optimization problems subject to bounds on variables. At the current iteration, the trial step is generated by the general trust-region subproblem which is defined by minimizing a quadratic function subject only to an affine scaling ellipsoidal constraint. Both trust-region strategy and line search filter technique will switch to trail backtracking step which is strictly feasible. Meanwhile, the proposed method does not depend on any external restoration procedure used in line search filter technique. A new backtracking relevance condition is given which is weaker than the switching condition to obtain the global convergence of the algorithm. The global convergence and fast local convergence rate of this algorithm are established under reasonable assumptions. Preliminary numerical results are reported indicating the practical viability and show the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper describes two optimal subgradient algorithms for solving structured large-scale convex constrained optimization. More specifically, the first algorithm is optimal for smooth problems with Lipschitz continuous gradients and for Lipschitz continuous nonsmooth problems, and the second algorithm is optimal for Lipschitz continuous nonsmooth problems. In addition, we consider two classes of problems: (i) a convex objective with a simple closed convex domain, where the orthogonal projection onto this feasible domain is efficiently available; and (ii) a convex objective with a simple convex functional constraint. If we equip our algorithms with an appropriate prox-function, then the associated subproblem can be solved either in a closed form or by a simple iterative scheme, which is especially important for large-scale problems. We report numerical results for some applications to show the efficiency of the proposed schemes. [ABSTRACT FROM AUTHOR]

In this paper, a subspace three-term conjugate gradient method is proposed. The search directions in the method are generated by minimizing a quadratic approximation of the objective function on a subspace. And they satisfy the descent condition and Dai-Liao conjugacy condition. At each iteration, the subspace is spanned by the current negative gradient and the latest two search directions. Thereby, the dimension of the subspace should be 2 or 3. Under some appropriate assumptions, the global convergence result of the proposed method is established. Numerical experiments show the proposed method is competitive for a set of 80 unconstrained optimization test problems. [ABSTRACT FROM AUTHOR]

In this paper, making use a exponential integral filter, a new algorithm for unconstrained global optimization is proposed. Compared with Yang's absolute value type integral filter method (Yang et al., Appl Math Comput 18:173-180, ), this algorithm is more effective and more sensitive. Numerical results for some typical examples show that in most cases, this algorithm works effectively and reliably. [ABSTRACT FROM AUTHOR]

Optimal homotopy analysis method is a powerful tool for nonlinear differential equations. In this method, the convergence of the series solutions is controlled by one or more parameters which can be determined by minimizing a certain function. There are several approaches to determine the optimal values of these parameters, which can be divided into two categories, i.e. global optimization approach and step-by-step optimization approach. In the global optimization approach, all the parameters are optimized simultaneously at the last order of approximation. However, this process leads to a system of coupled, nonlinear algebraic equations in multiple variables which are very difficult to solve. In the step-by-step approach, the optimal values of these parameters are determined sequentially, that is, they are determined one by one at different orders of approximation. In this way, the computational efficiency is significantly improved, especially when high order of approximation is needed. In this paper, we provide extensive examples arising in similarity and non-similarity boundary layer theory to investigate the performance of these approaches. The results reveal that with the step-by-step approach, convergent solutions of high order of approximation can be obtained within much less CPU time, compared with the global approach and the traditional HAM. [ABSTRACT FROM AUTHOR]

Practical Nonlinear Programming algorithms may converge to infeasible points. It is sensible to detect this situation as quickly as possible, in order to have time to change initial approximations and parameters, with the aim of obtaining convergence to acceptable solutions in further runs. In this paper, a recently introduced Augmented Lagrangian algorithm is modified in such a way that the probability of quick detection of asymptotic infeasibility is enhanced. The modified algorithm preserves the property of convergence to stationary points of the sum of squares of infeasibilities without harming the convergence to KKT points in feasible cases. [ABSTRACT FROM AUTHOR]

Kernel functions play an important role in the design and analysis of primal-dual interior-point algorithms. They are not only used for determining the search directions but also for measuring the distance between the given iterate and the μ-center for the algorithms. In this paper we present a unified kernel function approach to primal-dual interior-point algorithms for convex quadratic semidefinite optimization based on the Nesterov and Todd symmetrization scheme. The iteration bounds for large- and small-update methods obtained are analogous to the linear optimization case. Moreover, this unifies the analysis for linear, convex quadratic and semidefinite optimizations. [ABSTRACT FROM AUTHOR]

In this paper, we propose a full Nesterov-Todd (NT) step infeasible interior-point algorithm for convex quadratic symmetric cone optimization based on Euclidean Jordan algebra. The algorithm uses only one feasibility step in each main iteration. The complexity result coincides with the best-known iteration bound for infeasible interior-point methods. [ABSTRACT FROM AUTHOR]

In this paper we present a new steepest-descent type algorithm for convex optimization problems. Our algorithm pieces the unknown into sub-blocs of unknowns and considers a partial optimization over each sub-bloc. In quadratic optimization, our method involves Newton technique to compute the step-lengths for the sub-blocs resulting descent directions. Our optimization method is fully parallel and easily implementable, we first presents it in a general linear algebra setting, then we highlight its applicability to a parabolic optimal control problem, where we consider the blocs of unknowns with respect to the time dependency of the control variable. The parallel tasks, in the last problem, turn 'on' the control during a specific time-window and turn it 'off' elsewhere. We show that our algorithm significantly improves the computational time compared with recognized methods. Convergence analysis of the new optimal control algorithm is provided for an arbitrary choice of partition. Numerical experiments are presented to illustrate the efficiency and the rapid convergence of the method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a numerical method for solving the stationary points of mathematical programs constrained by parameterized quasi-variational inequalities. The necessary optimality conditions (stationary conditions in the sense of Mordukhovich) for the optimization problem are reformulated as a system of nonsmooth equations without the strict complementarity condition and an inexact Newton method is constructed to find its solutions. The local convergence of the inexact Newton method is guaranteed under second order sufficient conditions and linear independence constraint qualification. Several illustrative examples are provided. [ABSTRACT FROM AUTHOR]

A correction to the article "Comments on another hybrid conjugate gradient algorithm for unconstrained optimization by Andrei" that was published online on September 8, 2015 is presented.

In this paper, a novel numerical method is proposed to solve specific third order ODE on semi-infinite interval. These kinds of problems often occur in laminar boundary layer with temperature dependent viscosity. Runge-Kutta method incorporating with optimization techniques is used to solve the problem. First, the semi-infinite interval is transformed into a finite interval. Second, by converting the boundary value problem, with some initial and distributed unknowns, into an optimization problem, solving the original problem is limited to solving a multiobjective optimization problem. Third, we use shooting-Newton's method for solving this optimization problem. It is shown that the Falkner-Skan problem with constant surface temperature, that arise during the solution for the laminar forced convection heat transfer from wedges to flow, can be solved accurately and simultaneously by this strategy. Numerical results for different values of wedge angle and Prandtl number are presented, which are in good agreement with some of the successful provided solutions in the literature. [ABSTRACT FROM AUTHOR]

This paper concerns with a new nonmonotone strategy and its application to the line search approach for unconstrained optimization. It has been believed that nonmonotone techniques can improve the possibility of finding the global optimum and increase the convergence rate of the algorithms. We first introduce a new nonmonotone strategy which includes a convex combination of the maximum function value of some preceding successful iterates and the current function value. We then incorporate the proposed nonmonotone strategy into an inexact Armijo-type line search approach to construct a more relaxed line search procedure. The global convergence to first-order stationary points is subsequently proved and the R-linear convergence rate are established under suitable assumptions. Preliminary numerical results finally show the efficiency and the robustness of the proposed approach for solving unconstrained nonlinear optimization problems. [ABSTRACT FROM AUTHOR]

In this paper, a new nonmonotone inexact line search rule is proposed and applied to the trust region method for unconstrained optimization problems. In our line search rule, the current nonmonotone term is a convex combination of the previous nonmonotone term and the current objective function value, instead of the current objective function value . We can obtain a larger stepsize in each line search procedure and possess nonmonotonicity when incorporating the nonmonotone term into the trust region method. Unlike the traditional trust region method, the algorithm avoids resolving the subproblem if a trial step is not accepted. Under suitable conditions, global convergence is established. Numerical results show that the new method is effective for solving unconstrained optimization problems. [ABSTRACT FROM AUTHOR]

In the framework of multistart and local search algorithms that find the global minimum of a real function f( x), $ x \in S\subseteq R^n$, Gaviano et alias proposed a rule for deciding, as soon as a local minimum has been found, whether to perform or not a new local minimization. That rule was designed to minimize the average local computational cost $eval_1(\cdotp)$ required to move from the current local minimum to a new one. In this paper the expression of the cost $eval_2(\cdotp)$ of the entire process of getting a global minimum is found and investigated; it is shown that $eval_2(\cdotp)$ has among its components $eval_1(\cdotp)$ and can be only monotonically increasing or decreasing; that is, it exhibits the same property of $eval_1(\cdotp)$. Moreover, a counterexample is given that shows that the optimal values given by $eval_1(\cdotp)$ and $eval_2(\cdotp)$ might not agree. Further, computational experiments of a parallel algorithm that uses the above rule are carried out in a MatLab environment. [ABSTRACT FROM AUTHOR]

In this paper we present a deterministic method for tracing the Pareto frontier in non-linear bi-objective optimization problems with equality and inequality constraints. We reformulate the bi-objective optimization problem as a parametric single-objective optimization problem with an additional Normalized Normal Equality Constraint (NNEC) similar to the existing Normal Boundary Intersection (NBI) and the Normalized Normal Constraint method (NNC). By computing the so called Defining Initial Value Problem (DIVP) for segments of the Pareto front and solving a continuation problem with a standard integrator for ordinary differential equations (ODE) we can trace the Pareto front. We call the resulting approach ODE NNEC method and demonstrate numerically that it can yield the entire Pareto frontier to high accuracy. Moreover, due to event detection capabilities available for common ODE integrators, changes in the active constraints can be automatically detected. The features of the current algorithm are illustrated for two case studies whose ® code is available as Electronic Supplementary Material to this article. [ABSTRACT FROM AUTHOR]

Nonmonotone line search approach is a new technique for solving optimization problems. It relaxes the line search range and finds a larger step-size at each iteration, so as to possibly avoid local minimizer and run away from narrow curved valley. It is helpful to find the global minimizer of optimization problems. In this paper we develop a new modification of matrix-free nonmonotone Armijo line search and analyze the global convergence and convergence rate of the resulting method. We also address several approaches to estimate the Lipschitz constant of the gradient of objective functions that would be used in line search algorithms. Numerical results show that this new modification of Armijo line search is efficient for solving large scale unconstrained optimization problems. [ABSTRACT FROM AUTHOR]

In this paper, we propose a non-monotone line search multidimensional filter-SQP method for general nonlinear programming based on the Wächter-Biegler methods for nonlinear equality constrained programming. Under mild conditions, the global convergence of the new method is proved. Furthermore, with the non-monotone technique and second order correction step, it is shown that the proposed method does not suffer from the Maratos effect, so that fast local convergence to second order sufficient local solutions is achieved. Numerical results show that the new approach is efficient. [ABSTRACT FROM AUTHOR]

An accelerated hybrid conjugate gradient algorithm represents the subject of this paper. The parameter β k is computed as a convex combination of $\beta_k^{HS}$ (Hestenes and Stiefel, J Res Nat Bur Stand 49:409–436, ) and $\beta_k^{DY}$ (Dai and Yuan, SIAM J Optim 10:177–182, ), i.e. $\beta_k^C =\left({1-\theta_k}\right)\beta_k^{HS} + \theta_k \beta_k^{DY}$. The parameter θ k in the convex combinaztion is computed in such a way the direction corresponding to the conjugate gradient algorithm is the best direction we know, i.e. the Newton direction, while the pair ( s k, y k) satisfies the modified secant condition given by Li et al. (J Comput Appl Math 202:523–539, ) B k + 1 s k = z k, where $z_k =y_k +\left({{\eta_k} / {\left\| {s_k} \right\|^2}} \right)s_k$, $\eta_k =2\left( {f_k -f_{k+1}} \right)+\left( {g_k +g_{k+1}} \right)^Ts_k$, s k = x k + 1 − x k and y k = g k + 1 − g k. It is shown that both for uniformly convex functions and for general nonlinear functions the algorithm with strong Wolfe line search is globally convergent. The algorithm uses an acceleration scheme modifying the steplength α k for improving the reduction of the function values along the iterations. Numerical comparisons with conjugate gradient algorithms show that this hybrid computational scheme outperforms a variant of the hybrid conjugate gradient algorithm given by Andrei (Numer Algorithms 47:143–156, ), in which the pair ( s k, y k) satisfies the classical secant condition B k + 1 s k = y k, as well as some other conjugate gradient algorithms including Hestenes-Stiefel, Dai-Yuan, Polack-Ribière-Polyak, Liu-Storey, hybrid Dai-Yuan, Gilbert-Nocedal etc. A set of 75 unconstrained optimization problems with 10 different dimensions is being used (Andrei, Adv Model Optim 10:147–161, ). [ABSTRACT FROM AUTHOR]

Abstract  Recently, Roos (SIAM J Optim 16(4):1110–1136, 2006) presented a primal-dual infeasible interior-point algorithm that uses full-Newton steps and whose iteration bound coincides with the best known bound for infeasible interior-point algorithms. In the current paper we use a different feasibility step such that the definition of the feasibility step in Mansouri and Roos (Optim Methods Softw 22(3):519–530, 2007) is a special case of our definition, and show that the same result on the order of iteration complexity can be obtained. [ABSTRACT FROM AUTHOR]

Direct multisearch (DMS) is a derivative-free optimization class of algorithms, suited for computing approximations to the complete Pareto front of a given multiobjective optimization problem. In DMS class, constraints are addressed with an extreme barrier approach, only evaluating feasible points. It has a well-supported convergence analysis and simple implementations present a good numerical performance, both in academic test sets and in real applications. Recently, this numerical performance was improved with the definition of a search step based on the minimization of quadratic polynomial models, corresponding to the algorithm BoostDMS. In this work, we propose and numerically evaluate strategies to improve the performance of BoostDMS, mainly through parallelization applied to the search and to the poll steps. The final parallelized version not only considerably decreases the computational time required for solving a multiobjective optimization problem, but also increases the quality of the computed approximation to the Pareto front. Extensive numerical results will be reported in an academic test set and in a chemical engineering application. [ABSTRACT FROM AUTHOR]

This work builds upon the theoretical and numerical results of the recently proposed Penalized Algorithm for Constrained Nonsmooth Optimization (PACNO). Our contribution is threefold. Instead of resting upon approximate stationary points of the subproblems, approximate local minimizers are assumed to be computed. Consequently, a stronger convergence result is obtained, based on a new sequential optimality condition. Moreover, using a blackbox minimization framework and hard-sphere instances, the intrinsic parameters of PACNO have been adjusted, improving outcomes from the literature for the kissing problem, which consists of determining the maximum number of non-overlapping and equal spheres that can touch simultaneously a given sphere of the same size. Finally, the so-called double-kissing problem has been modeled: two equal and touching spheres are provided, and one aims at finding the maximum number of non-overlapping spheres, having the same radius of the given pair, which can be arranged so that each of them touches at least one of the stated spheres. A nonsmooth formulation for the double-kissing problem is devised, and the solutions of bi-, three-, and four-dimensional instances are successfully achieved. [ABSTRACT FROM AUTHOR]

In this article, we consider an optimization problem where the objective function is evaluated at the fixed-point of a contraction mapping parameterized by a control variable, and optimization takes place over this control variable. Since the derivative of the fixed-point with respect to the parameter can usually not be evaluated exactly, an adjoint dynamical system can be used to estimate gradients. Using this estimation procedure, the optimization algorithm alternates between derivative estimation and an approximate gradient descent step. We analyze a variant of this approach involving dynamic time-scaling, where after each parameter update the adjoint system is iterated until a convergence threshold is passed. We prove that, under certain conditions, the algorithm can find approximate stationary points of the objective function. We demonstrate the approach in the settings of an inverse problem in chemical kinetics, and learning in attractor networks. [ABSTRACT FROM AUTHOR]

In this article, based on the modified secant equation, we propose a modified Hestenes-Stiefel (HS) conjugate gradient method which has similar form as the CG-DESCENT method proposed by Hager and Zhang (SIAM J Optim 16:170-192, ). The presented method can generate sufficient descent directions without any line search. Under some mild conditions, we show that it is globally convergent with Armijo line search. Moreover, the R-linear convergence rate of the modified HS method is established. Preliminary numerical results show that the proposed method is promising, and competitive with the well-known CG-DESCENT method. [ABSTRACT FROM AUTHOR]