Search

In the past several years, augmented Lagrangian methods have been successfully applied to several classes of nonconvex optimization problems, inspiring new developments in both theory and practice. In this paper we bring most of these recent developments from nonlinear programming to the context of optimization on Riemannian manifolds, including equality and inequality constraints. Many research have been conducted on optimization problems on manifolds, however only recently the treatment of the constrained case has been considered. In this paper we propose to bridge this gap with respect to the most recent developments in nonlinear programming. In particular, we formulate several well-known constraint qualifications from the Euclidean context which are sufficient for guaranteeing global convergence of augmented Lagrangian methods, without requiring boundedness of the set of Lagrange multipliers. Convergence of the dual sequence can also be assured under a weak constraint qualification. The theory presented is based on so-called sequential optimality conditions, which is a powerful tool used in this context. The paper can also be read with the Euclidean context in mind, serving as a review of the most relevant constraint qualifications and global convergence theory of state-of-the-art augmented Lagrangian methods for nonlinear programming. [ABSTRACT FROM AUTHOR]

Nonlinear symmetric cone programming (NSCP) generalizes important optimization problems such as nonlinear programming, nonlinear semi-definite programming and nonlinear second-order cone programming (NSOCP). In this work, we present two new optimality conditions for NSCP without constraint qualifications, which implies the Karush–Kuhn–Tucker conditions under a condition weaker than Robinson's constraint qualification. In addition, we show the relationship of both optimality conditions in the context of NSOCP, where we also present an augmented Lagrangian method with global convergence to a KKT point under a condition weaker than Robinson's constraint qualification. [ABSTRACT FROM AUTHOR]

The well known constant rank constraint qualification [Math. Program. Study 21:110–126, 1984] introduced by Janin for nonlinear programming has been recently extended to a conic context by exploiting the eigenvector structure of the problem. In this paper we propose a more general and geometric approach for defining a new extension of this condition to the conic context. The main advantage of our approach is that we are able to recast the strong second-order properties of the constant rank condition in a conic context. In particular, we obtain a second-order necessary optimality condition that is stronger than the classical one obtained under Robinson's constraint qualification, in the sense that it holds for every Lagrange multiplier, even though our condition is independent of Robinson's condition. [ABSTRACT FROM AUTHOR]

Second-order necessary optimality conditions for nonlinear conic programming problems that depend on a single Lagrange multiplier are usually built under nondegeneracy and strict complementarity. In this paper we establish a condition of such type for two classes of nonlinear conic problems, namely semidefinite and second-order cone programming, assuming Robinson’s constraint qualification and a weak constant rank-type property which are, together, strictly weaker than nondegeneracy. Our approach is done via a penalty-based strategy, which is aimed at providing strong global convergence results for first- and second-order algorithms. Since we are not assuming strict complementarity, the critical cone does not reduce to a subspace, thus, the second-order condition we arrive at is defined in terms of the lineality space of the critical cone. In the case of nonlinear programming, this condition reduces to the standard second-order condition widely used as second-order stationarity measure in the algorithmic practice. [ABSTRACT FROM AUTHOR]

We present new constraint qualification conditions for nonlinear semidefinite programming that extend some of the constant rank-type conditions from nonlinear programming. As an application of these conditions, we provide a unified global convergence proof of a class of algorithms to stationary points without assuming neither uniqueness of the Lagrange multiplier nor boundedness of the Lagrange multipliers set. This class of algorithms includes, for instance, general forms of augmented Lagrangian, sequential quadratic programming, and interior point methods. In particular, we do not assume boundedness of the dual sequence generated by the algorithm. The weaker sequential condition we present is shown to be strictly weaker than Robinson’s condition while still implying metric subregularity. [ABSTRACT FROM AUTHOR]

In Andreani et al. (Weak notions of nondegeneracy in nonlinear semidefinite programming, 2020), the classical notion of nondegeneracy (or transversality) and Robinson's constraint qualification have been revisited in the context of nonlinear semidefinite programming exploiting the structure of the problem, namely its eigendecomposition. This allows formulating the conditions equivalently in terms of (positive) linear independence of significantly smaller sets of vectors. In this paper, we extend these ideas to the context of nonlinear second-order cone programming. For instance, for an m-dimensional second-order cone, instead of stating nondegeneracy at the vertex as the linear independence of m derivative vectors, we do it in terms of several statements of linear independence of 2 derivative vectors. This allows embedding the structure of the second-order cone into the formulation of nondegeneracy and, by extension, Robinson's constraint qualification as well. This point of view is shown to be crucial in defining significantly weaker constraint qualifications such as the constant rank constraint qualification and the constant positive linear dependence condition. Also, these conditions are shown to be sufficient for guaranteeing global convergence of several algorithms, while still implying metric subregularity and without requiring boundedness of the set of Lagrange multipliers. [ABSTRACT FROM AUTHOR]

Sequential optimality conditions play a major role in proving stronger global convergence results of numerical algorithms for nonlinear programming. Several extensions are described in conic contexts, in which many open questions have arisen. In this paper, we present new sequential optimality conditions in the context of a general nonlinear conic framework, which explains and improves several known results for specific cases, such as semidefinite programming, second-order cone programming, and nonlinear programming. In particular, we show that feasible limit points of sequences generated by the augmented Lagrangian method satisfy the so-called approximate gradient projection optimality condition and, under an additional smoothness assumption, the so-called complementary approximate Karush–Kuhn–Tucker condition. The first result was unknown even for nonlinear programming, and the second one was unknown, for instance, for semidefinite programming. [ABSTRACT FROM AUTHOR]

The optimization literature is vast in papers dealing with improvements on the global convergence of augmented Lagrangian schemes. Usually, the results are based on weak constraint qualifications, or, more recently, on sequential optimality conditions obtained via penalization techniques. In this paper, we propose a somewhat different approach, in the sense that the algorithm itself is used in order to formulate a new optimality condition satisfied by its feasible limit points. With this tool at hand, we present several new properties and insights on limit points of augmented Lagrangian schemes, in particular, characterizing the strongest possible global convergence result for the safeguarded augmented Lagrangian method. [ABSTRACT FROM AUTHOR]

In this note we present a sufficient condition for an upper bound on the rank of a set { A 1 , A 2 , ... , A m } of real n × n symmetric matrices to hold. The condition is based on the rank of a matrix defined by vectors in the image of each matrix, namely A 1 x , A 2 x , ... , A m x , x ∈ R n together with an auxiliary matrix V ∈ R k × m . The result is inspired by some problems on second-order necessary optimality conditions for constrained optimization with quadratic constraints, and we explain and unify previously known results. [ABSTRACT FROM AUTHOR]

We analyze sequences generated by interior point methods (IPMs) in convex and nonconvex settings. We prove that moving the primal feasibility at the same rate as the barrier parameter μ ensures the Lagrange multiplier sequence remains bounded, provided the limit point of the primal sequence has a Lagrange multiplier. This result does not require constraint qualifications. We also guarantee the IPM finds a solution satisfying strict complementarity if one exists. On the other hand, if the primal feasibility is reduced too slowly, then the algorithm converges to a point of minimal complementarity; if the primal feasibility is reduced too quickly and the set of Lagrange multipliers is unbounded, then the norm of the Lagrange multiplier tends to infinity. Our theory has important implications for the design of IPMs. Specifically, we show that IPOPT, an algorithm that does not carefully control primal feasibility has practical issues with the dual multipliers values growing to unnecessarily large values. Conversely, the one-phase IPM of Hinder and Ye (A one-phase interior point method for nonconvex optimization, 2018. arXiv:1801.03072), an algorithm that controls primal feasibility as our theory suggests, has no such issue. [ABSTRACT FROM AUTHOR]

The notion of a normal cone of a given set is paramount in optimization and variational analysis. In this work, we give a definition of a multiobjective normal cone, which is suitable for studying optimality conditions and constraint qualifications for multiobjective optimization problems. A detailed study of the properties of the multiobjective normal cone is conducted. With this tool, we were able to characterize weak and strong Karush–Kuhn–Tucker conditions by means of a Guignard-type constraint qualification. Furthermore, the computation of the multiobjective normal cone under the error bound property is provided. The important statements are illustrated by examples. [ABSTRACT FROM AUTHOR]

Interior point methods have attracted most of the attention in the recent decades for solving large scale convex quadratic programming problems. In this paper we take a different route as we present an augmented Lagrangian method for convex quadratic programming based on recent developments for nonlinear programming. In our approach, box constraints are penalized while equality constraints are kept within the subproblems. The motivation for this approach is that Newton's method can be efficient for minimizing a piecewise quadratic function. Moreover, since augmented Lagrangian methods do not rely on proximity to the central path, some of the inherent difficulties in interior point methods can be avoided. In addition, a good starting point can be easily exploited, which can be relevant for solving subproblems arising from sequential quadratic programming, in sensitivity analysis and in branch and bound techniques. We prove well-definedness and finite convergence of the method proposed. Numerical experiments on separable strictly convex quadratic problems formulated from the Netlib collection show that our method can be competitive with interior point methods, in particular when a good initial point is available and a second-order Lagrange multiplier update is used. [ABSTRACT FROM AUTHOR]

Sequential optimality conditions have played a major role in unifying and extending global convergence results for several classes of algorithms for general nonlinear optimization. In this paper, we extend theses concepts for nonlinear semidefinite programming. We define two sequential optimality conditions for nonlinear semidefinite programming. The first is a natural extension of the so-called Approximate-Karush–Kuhn–Tucker (AKKT), well known in nonlinear optimization. The second one, called Trace-AKKT, is more natural in the context of semidefinite programming as the computation of eigenvalues is avoided. We propose an augmented Lagrangian algorithm that generates these types of sequences and new constraint qualifications are proposed, weaker than previously considered ones, which are sufficient for the global convergence of the algorithm to a stationary point. [ABSTRACT FROM AUTHOR]

In this work, we present an Augmented Lagrangian algorithm for nonlinear semidefinite problems (NLSDPs), which is a natural extension of its consolidated counterpart in nonlinear programming. This method works with two levels of constraints; one that is penalized and other that is kept within the subproblems. This is done to allow exploiting the subproblem structure while solving it. The global convergence theory is based on recent results regarding approximate Karush–Kuhn–Tucker optimality conditions for NLSDPs, which are stronger than the usually employed Fritz John optimality conditions. Additionally, we approach the problem of covering a given object with a fixed number of balls with a minimum radius, where we employ some convex algebraic geometry tools, such as Stengle's Positivstellensatz and its variations, which allows for a much more general model. Preliminary numerical experiments are presented. [ABSTRACT FROM AUTHOR]

In this paper we consider the minimization of a continuous function that is potentially not differentiable or not twice differentiable on the boundary of the feasible region. By exploiting an interior point technique, we present first- and second-order optimality conditions for this problem that reduces to classical ones when the derivative on the boundary is available. For this type of problems, existing necessary conditions often rely on the notion of subdifferential or become non-trivially weaker than the KKT condition in the (twice-)differentiable counterpart problems. In contrast, this paper presents a new set of first- and second-order necessary conditions that are derived without the use of subdifferential and reduce to exactly the KKT condition when (twice-)differentiability holds. As a result, these conditions are stronger than some existing ones considered for the discussed minimization problem when only non-negativity constraints are present. To solve for these optimality conditions in the special but important case of linearly constrained problems, we present two novel interior point trust-region algorithms and show that their worst-case computational efficiency in achieving the potentially stronger optimality conditions match the best known complexity bounds. Since this work considers a more general problem than those in the literature, our results also indicate that best known existing complexity bounds are actually held for a wider class of nonlinear programming problems. This new development is significant since optimality conditions play a fundamental role in computational optimization and more and more nonlinear and nonconvex problems need to be solved in practice. [ABSTRACT FROM AUTHOR]

Generalized Nash equilibrium problems (GNEPs) are a generalization of the classic Nash equilibrium problems (NEPs), where each player's strategy set depends on the choices of the other players. In this work we study constraint qualifcations (CQs) and optimality conditions tailored for GNEPs, and we discuss their relations and implications for global convergence of algorithms. We show the surprising fact that, in contrast to the case of nonlinear programming, in general the Karush-Kuhn-Tucker (KKT) residual cannot be made arbitrarily small near a solution of a GNEP. We then discuss some important practical consequences of this fact. We also prove that this phenomenon is not present in an important class of GNEPs, including NEPs. Finally, under an introduced weak CQ, we prove global convergence to a KKT point of an augmented Lagrangian algorithm for GNEPs, and under the quasi-normality (QN) CQ for GNEPs, we prove boundedness of the dual sequence. [ABSTRACT FROM AUTHOR]

Generalized Nash equilibrium problems (GNEPs) are a generalization of the classic Nash equilibrium problems (NEPs), where each player's strategy set depends on the choices of the other players. In this work we study constraint qualifications (CQs) and optimality conditions tailored for GNEPs, and we discuss their relations and implications for global convergence of algorithms. We show the surprising fact that, in contrast to the case of nonlinear programming, in general the Karush--Kuhn--Tucker (KKT) residual cannot be made arbitrarily small near a solution of a GNEP. We then discuss some important practical consequences of this fact. We also prove that this phenomenon is not present in an important class of GNEPs, including NEPs. Finally, under an introduced weak CQ, we prove global convergence to a KKT point of an augmented Lagrangian algorithm for GNEPs, and under the quasi-normality (QN) CQ for GNEPs, we prove boundedness of the dual sequence. [ABSTRACT FROM AUTHOR]

We develop a new notion of second-order complementarity with respect to the tangent subspace related to second-order necessary optimality conditions by the introduction of so-called tangent multipliers. We prove that around a local minimizer, a second-order stationarity residual can be driven to zero while controlling the growth of Lagrange multipliers and tangent multipliers, which gives a new second-order optimality condition without constraint qualifications stronger than previous ones associated with global convergence of algorithms. We prove that second-order variants of augmented Lagrangian (under an additional smoothness assumption based on the Lojasiewicz inequality) and interior point methods generate sequences satisfying our optimality condition. We present also a companion minimal constraint qualification, weaker than the ones known for second-order methods, that ensures usual global convergence results to a classical second-order stationary point. Finally, our optimality condition naturally suggests a definition of second-order stationarity suitable for the computation of iteration complexity bounds and for the definition of stopping criteria. [ABSTRACT FROM AUTHOR]

In this paper, we deal with a conjecture formulated in Andreani et al. (Optimization 56:529-542, 2007), which states that whenever a local minimizer of a nonlinear optimization problem fulfills the Mangasarian-Fromovitz constraint qualification and the rank of the set of gradients of active constraints increases at most by one in a neighborhood of the minimizer, a second-order optimality condition that depends on one single Lagrange multiplier is satisfied. This conjecture generalizes previous results under a constant rank assumption or under a rank deficiency of at most one. We prove the conjecture under the additional assumption that the Jacobian matrix has a smooth singular value decomposition. Our proof also extends to the case of the strong second-order condition, defined in terms of the critical cone instead of the critical subspace. [ABSTRACT FROM AUTHOR]

Sequential optimality conditions have recently played an important role on the analysis of the global convergence of optimization algorithms towards first-order stationary points, justifying their stopping criteria. In this article, we introduce a sequential optimality condition that takes into account second-order information and that allows us to improve the global convergence assumptions of several second-order algorithms, which is our main goal. We also present a companion constraint qualification that is less stringent than previous assumptions associated to the convergence of second-order methods, like the joint condition Mangasarian-Fromovitz and weak constant rank. Our condition is also weaker than the constant rank constraint qualification. This means that we can prove second-order global convergence of well-established algorithms even when the set of Lagrange multipliers is unbounded, which was a limitation of previous results based on Mangasarian-Fromovitz constraint qualification. We prove global convergence of well-known variations of the augmented Lagrangian and regularized sequential quadratic programming methods to second-order stationary points under this new weak constraint qualification. [ABSTRACT FROM AUTHOR]

We prove an extension of Yuan's lemma to more than two matrices, as long as the set of matrices has rank at most 2. This is used to generalize the main result of Baccari and Trad (SIAM J Optim 15(2):394-408, 2005), where the classical necessary second-order optimality condition is proved, under the assumption that the set of Lagrange multipliers is a bounded line segment. We prove the result under the more general assumption that the Hessian of the Lagrangian, evaluated at the vertices of the Lagrange multiplier set, is a matrix set with at most rank 2. We apply the results to prove the classical second-order optimality condition to problems with quadratic constraints and without constant rank of the Jacobian matrix. [ABSTRACT FROM AUTHOR]

In this work we present new weak conditions that ensure the validity of necessary second-order optimality conditions (SOC) for nonlinear optimization. We are able to prove that weak and strong SOCs hold for all Lagrange multipliers using Abadie-type assumptions. We also prove weak and strong SOCs for at least one Lagrange multiplier imposing the Mangasarian–Fromovitz constraint qualification and a weak constant rank assumption. [ABSTRACT FROM AUTHOR]

A Correction to this paper has been published: https://doi.org/10.1007/s11075-021-01241-3 [ABSTRACT FROM AUTHOR]

We consider the minimization of a convex function on a bounded polyhedron (polytope) represented by linear equality constraints and non-negative variables. We define the Levenberg-Marquardt and central trajectories starting at the analytic center using the same parameter, and show that they satisfy a primal-dual relationship, being close to each other for large values of the parameter. Based on this, we develop an algorithm that starts computing primal-dual feasible points on the Levenberg-Marquardt trajectory and eventually moves to the central path. Our main theorem is particularly relevant in quadratic programming, where points on the primal-dual Levenberg-Marquardt trajectory can be calculated by means of a system of linear equations. We present some computational tests related to box constrained trust region subproblems. [ABSTRACT FROM AUTHOR]

In this work we introduce a relaxed version of the constant positive linear dependence constraint qualification (CPLD) that we call RCPLD. This development is inspired by a recent generalization of the constant rank constraint qualification by Minchenko and Stakhovski that was called RCRCQ. We show that RCPLD is enough to ensure the convergence of an augmented Lagrangian algorithm and that it asserts the validity of an error bound. We also provide proofs and counter-examples that show the relations of RCRCQ and RCPLD with other known constraint qualifications. In particular, RCPLD is strictly weaker than CPLD and RCRCQ, while still stronger than Abadie's constraint qualification. We also verify that the second order necessary optimality condition holds under RCRCQ. [ABSTRACT FROM AUTHOR]

We present two new constraint qualifications (CQs) that are weaker than the recently introduced relaxed constant positive linear dependence (RCPLD) CQ. RCPLD is based on the assumption that many subsets of the gradients of the active constraints preserve positive linear dependence locally. A major open question was to identify the exact set of gradients whose properties had to be preserved locally and that would still work as a CQ. This is done in the first new CQ, which we call the constant rank of the subspace component (CRSC) CQ. This new CQ also preserves many of the good properties of RCPLD, such as local stability and the validity of an error bound. We also introduce an even weaker CQ, called the constant positive generator (CPG), which can replace RCPLD in the analysis of the global convergence of algorithms. We close this work by extending convergence results of algorithms belonging to all the main classes of nonlinear optimization methods: sequential quadratic programming, augmented Lagrangians, interior point algorithms, and inexact restoration. [ABSTRACT FROM AUTHOR]

In this work, we introduce a necessary sequential Approximate-Karush-Kuhn-Tucker (AKKT) condition for a point to be a solution of a continuous variational inequality, and we prove its relation with the Approximate Gradient Projection condition (AGP) of Gárciga-Otero and Svaiter. We also prove that a slight variation of the AKKT condition is sufficient for a convex problem, either for variational inequalities or optimization. Sequential necessary conditions are more suitable to iterative methods than usual punctual conditions relying on constraint qualifications. The AKKT property holds at a solution independently of the fulfillment of a constraint qualification, but when a weak one holds, we can guarantee the validity of the KKT conditions. [ABSTRACT FROM AUTHOR]

Sequential optimality conditions provide adequate theoretical tools to justify stopping criteria for nonlinear programming solvers. Approximate Karush-Kuhn-Tucker and approximate gradient projection conditions are analysed in this work. These conditions are not necessarily equivalent. Implications between different conditions and counter-examples will be shown. Algorithmic consequences will be discussed. [ABSTRACT FROM AUTHOR]

Carathéodory's lemma states that if we have a linear combination of vectors in ℝn, we can rewrite this combination using a linearly independent subset. This lemma has been successfully applied in nonlinear optimization in many contexts. In this work we present a new version of this celebrated result, in which we obtained new bounds for the size of the coefficients in the linear combination and we provide examples where these bounds are useful. We show how these new bounds can be used to prove that the internal penalty method converges to KKT points, and we prove that the hypothesis to obtain this result cannot be weakened. The new bounds also provides us some new results of convergence for the quasi feasible interior point `2-penalty method of Chen and Goldfarb [7]. [ABSTRACT FROM AUTHOR]

Sequential optimality conditions provide adequate theoretical tools to justify stopping criteria for nonlinear programming solvers. Approximate Karush–Kuhn–Tucker and approximate gradient projection conditions are analysed in this work. These conditions are not necessarily equivalent. Implications between different conditions and counter-examples will be shown. Algorithmic consequences will be discussed. [ABSTRACT FROM PUBLISHER]