Search

We propose an algorithm for general nonlinear conic programming which does not require the knowledge of the full cone, but rather a simpler, more tractable, approximation of it. We prove that the algorithm satisfies a strong global convergence property in the sense that it generates a strong sequential optimality condition. In particular, a KKT point is necessarily found when a limit point satisfies Robinson's condition. We conduct numerical experiments minimizing nonlinear functions subject to a copositive cone constraint. In order to do this, we consider a well known polyhedral approximation of this cone by means of refining the polyhedral constraints after each augmented Lagrangian iteration. We show that our strategy outperforms the standard approach of considering a close polyhedral approximation of the full copositive cone in every iteration., Comment: 13 pages, 5 figures

In the seminal book M\'echanique analitique, Lagrange, 1788, the notion of a Lagrange multiplier was first introduced in order to study a smooth minimization problem subject to equality constraints. The idea is that, under some regularity assumption, at a solution of the problem, one may associate a new variable (Lagrange multiplier) to each constraint such that an equilibrium equation is satisfied. This concept turned out to be central in studying more general constrained optimization problems and it has lead to the rapid development of nonlinear programming as a field of mathematics since the works of Karush and Kuhn-Tucker, who considered equality and inequality constraints. The usual proofs for the existence of Lagrange multipliers are somewhat cumbersome, relying on the implicit function theorem or duality theory. In the first section of this note we present an elementary proof of existence of Lagrange multipliers in the simplest context, which is easily accessible to a wide variety of readers. In addition, this proof is readily extended to the much more general context of conic constraints, which we present in the second section together with the background properties needed on the projection onto a closed and convex cone.

In this paper, we extend and investigate the properties of the semi-smooth Newton method when applied to a general projection equation in finite dimensional spaces. We first present results concerning Clarke's generalized Jacobian of the projection onto a closed and convex cone. We then describe the iterative process for the general cone case and establish two convergence theorems. We apply these results to the constrained quadratic conic programming problem, emphasizing its connection to the projection equation. To illustrate the performance of our method, we conduct numerical experiments focusing on semidefinite least squares, in particular the nearest correlation matrix problem. In the latter scenario, we benchmark our outcomes against previous literature, presenting performance profiles and tabulated results for clarity and comparison.

In the past years, augmented Lagrangian methods have been successfully applied to several classes of non-convex optimization problems, inspiring new developments in both theory and practice. In this paper we bring most of these recent developments from nonlinear programming to the context of optimization on Riemannian manifolds, including equality and inequality constraints. Many research have been conducted on optimization problems on manifolds, however only recently the treatment of the constrained case has been considered. In this paper we propose to bridge this gap with respect to the most recent developments in nonlinear programming. In particular, we formulate several well known constraint qualifications from the Euclidean context which are sufficient for guaranteeing global convergence of augmented Lagrangian methods, without requiring boundedness of the set of Lagrange multipliers. Convergence of the dual sequence can also be assured under a weak constraint qualification. The theory presented is based on so called sequential optimality conditions, which is a powerful tool used in this context. The paper can also be read with the Euclidean context in mind, serving as a review of the most relevant constraint qualifications and global convergence theory of state-of-the-art augmented Lagrangian methods for nonlinear programming., Comment: 25 pages

In a previous paper [R. Andreani, G. Haeser, L. M. Mito, H. Ram\'irez, T. P. Silveira. First- and second-order optimality conditions for second-order cone and semidefinite programming under a constant rank condition. Mathematical Programming, 2023. DOI: 10.1007/s10107-023-01942-8] we introduced a constant rank constraint qualification for nonlinear semidefinite and second-order cone programming by considering all faces of the underlying cone. This condition is independent of Robinson's condition and it implies a strong second-order necessary optimality condition which depends on a single Lagrange multiplier instead of the full set of Lagrange multipliers. In this paper we expand on this result in several directions, namely, we consider the larger class of $\mathcal{C}^2-$cone reducible constraints and we show that it is not necessary to consider all faces of the cone; instead a single specific face should be considered (which turns out to be weaker than Robinson's condition) in order for the first order necessary optimality condition to hold. This gives rise to a notion of facial reduction for nonlinear conic programming, that allows locally redefining the original problem only in terms of this specific face instead of the whole cone, providing a more robust formulation of the problem in which Robinson's condition holds. We were also able to prove the strong second-order necessary optimality condition in this context by considering only the subfaces of this particular face, which is a new result even in nonlinear programming., Comment: 16 pages

A common strategy for solving an unconstrained two-player Nash equilibrium problem with continuous variables is applying Newton's method to the system of nonlinear equations obtained by the corresponding first-order necessary optimality conditions. However, when taking into account the game dynamics, it is not clear what is the goal of each player when considering that they are taking their current decision following Newton's iterates. In this paper we provide an interpretation for Newton's iterate in view of the game dynamics as follows: instead of minimizing the quadratic approximation of their objective function parameterized by the other player current decision (as a typical Jacobi-type strategy), we show that the Newton iterate follows this approach but with the objective function parameterized by a prediction of the other player action, considering that they are following the same Newtonian strategy. This interpretation allows us to present a new Newtonian algorithm where a backtracking procedure is introduced in order to guarantee that the computed Newtonian directions, for each player, are descent directions for their corresponding parameterized functions. Thus, besides favoring global convergence, our algorithm also favors true minimizers instead of maximizers or saddle points, differently from the standard Newton method, which does not consider the minimization structure of the problem in the non-convex case. Thus, our method is more robust in comparison with other Jacobi-type strategies or the pure Newtonian approach, which is corroborated by our illustrative numerical experiments. We also present a proof of the well-definiteness of the algorithm under some standard assumptions, together with a thorough analysis of its convergence properties taking into account the game dynamics., Comment: 3 figures

Second-order necessary optimality conditions for nonlinear conic programming problems that depend on a single Lagrange multiplier are usually built under nondegeneracy and strict complementarity. In this paper we establish a condition of such type for two classes of nonlinear conic problems, namely semidefinite and second-order cone programming, assuming Robinson's constraint qualification and a weak constant rank-type property which are, together, strictly weaker than nondegeneracy. Our approach is done via a penalty-based strategy, which is aimed at providing strong global convergence results for first- and second-order algorithms. Since we are not assuming strict complementarity, the critical cone does not reduce to a subspace, thus, the second-order condition we arrive at is defined in terms of the lineality space of the critical cone. In the case of nonlinear programming, this condition reduces to the standard second-order condition widely used as second-order stationarity measure in the algorithmic practice., Comment: 20 pages

This paper is devoted to studying the global and finite convergence of the semi-smooth Newton method for solving a piecewise linear system that arises in cone-constrained quadratic programming problems and absolute value equations. We first provide a negative answer via a counterexample to a conjecture on the global and finite convergence of the Newton iteration for symmetric and positive definite matrices. Additionally, we discuss some surprising features of the semi-smooth Newton iteration in low dimensions and its behavior in higher dimensions. Moreover, we present two iterative schemes inspired by the classical Jacobi and Gauss-Seidel methods for linear systems of equations for finding a solution to the problem. We study sufficient conditions for the convergence of both proposed procedures, which are also sufficient for the existence and uniqueness of solutions to the problem. Lastly, we perform some computational experiments designed to illustrate the behavior (in terms of CPU time) of the proposed iterations versus the semi-smooth Newton method for dense and sparse large-scale problems.

In [R. Andreani, G. Haeser, L. M. Mito, H. Ram\'irez C., Weak notions of nondegeneracy in nonlinear semidefinite programming, arXiv:2012.14810, 2020] the classical notion of nondegeneracy (or transversality) and Robinson's constraint qualification have been revisited in the context of nonlinear semidefinite programming exploiting the structure of the problem, namely, its eigendecomposition. This allows formulating the conditions equivalently in terms of (positive) linear independence of significantly smaller sets of vectors. In this paper we extend these ideas to the context of nonlinear second-order cone programming. For instance, for an $m$-dimensional second-order cone, instead of stating nondegeneracy at the vertex as the linear independence of $m$ derivative vectors, we do it in terms of several statements of linear independence of $2$ derivative vectors. This allows embedding the structure of the second-order cone into the formulation of nondegeneracy and, by extension, Robinson's constraint qualification as well. This point of view is shown to be crucial in defining significantly weaker constraint qualifications such as the constant rank constraint qualification and the constant positive linear dependence condition. Also, these conditions are shown to be sufficient for guaranteeing global convergence of several algorithms, while still implying metric subregularity and without requiring boundedness of the set of Lagrange multipliers., Comment: 21 pages. Corrections

The well known constant rank constraint qualification [Math. Program. Study 21:110--126, 1984] introduced by Janin for nonlinear programming has been recently extended to a conic context by exploiting the eigenvector structure of the problem. In this paper we propose a more general and geometric approach for defining a new extension of this condition to the conic context. The main advantage of our approach is that we are able to recast the strong second-order properties of the constant rank condition in a conic context. In particular, we obtain a second-order necessary optimality condition that is stronger than the classical one obtained under Robinson's constraint qualification, in the sense that it holds for every Lagrange multiplier, even though our condition is independent of Robinson's condition., Comment: 27 pages, 1 figure

We present new constraint qualification conditions for nonlinear semidefinite programming that extend some of the constant rank-type conditions from nonlinear programming. As an application of these conditions, we provide a unified global convergence proof of a class of algorithms to stationary points without assuming neither uniqueness of the Lagrange multiplier nor boundedness of the Lagrange multipliers set. This class of algorithm includes, for instance, general forms of augmented Lagrangian, sequential quadratic programming, and interior point methods. We also compare these new conditions with some of the existing ones, including the nondegeneracy condition, Robinson's constraint qualification, and the metric subregularity constraint qualification., Comment: 19 pages, trimmed

The constraint nondegeneracy condition is one of the most relevant and useful constraint qualifications in nonlinear semidefinite programming. It can be characterized in terms of any fixed orthonormal basis of the, let us say, $\ell$-dimensional kernel of the constraint matrix, by the linear independence of a set of $\ell(\ell+1)/2$ derivative vectors. We show that this linear independence requirement can be equivalently formulated in a smaller set, of $\ell$ derivative vectors, by considering all orthonormal bases of the kernel instead. This allows us to identify that not all bases are relevant for a constraint qualification to be defined, giving rise to a strictly weaker variant of nondegeneracy related to the global convergence of an external penalty method. We use some of these ideas to revisit an approach of Forsgren [Math. Prog. 88, 105--128, 2000] for exploiting the sparsity structure of a transformation of the constraints to define a constraint qualification, which led us to develop another relaxed notion of nondegeneracy using a simpler transformation. If the zeros of the derivatives of the constraint function at a given point are considered, instead of the zeros of the function itself in a neighborhood of that point, we obtain an even weaker constraint qualification that connects Forsgren's condition and ours., Comment: 17 pages, 1 figure. Some results have been revised. GS-nondegeneracy added

We analyze sequences generated by interior point methods (IPMs) in convex and nonconvex settings. We prove that moving the primal feasibility at the same rate as the barrier parameter $\mu$ ensures the Lagrange multiplier sequence remains bounded, provided the limit point of the primal sequence has a Lagrange multiplier. This result does not require constraint qualifications. We also guarantee the IPM finds a solution satisfying strict complementarity if one exists. On the other hand, if the primal feasibility is reduced too slowly, then the algorithm converges to a point of minimal complementarity; if the primal feasibility is reduced too quickly and the set of Lagrange multipliers is unbounded, then the norm of the Lagrange multiplier tends to infinity. Our theory has important implications for the design of IPMs. Specifically, we show that IPOPT, an algorithm that does not carefully control primal feasibility has practical issues with the dual multipliers values growing to unnecessarily large values. Conversely, the one-phase IPM of \citet*{hinder2018one}, an algorithm that controls primal feasibility as our theory suggests, has no such issue.

We prove an extension of Yuan's Lemma to more than two matrices, as long as the set of matrices has rank at most 2. This is used to generalize the main result of [A. Baccari and A. Trad. On the classical necessary second-order optimality conditions in the presence of equality and inequality constraints. SIAM J. Opt., 15(2):394--408, 2005], where the classical necessary second-order optimality condition is proved under the assumption that the set of Lagrange multipliers is a bounded line segment. We prove the result under the more general assumption that the hessian of the Lagrangian evaluated at the vertices of the Lagrange multiplier set is a matrix set with at most rank 2. We apply the results to prove the classical second-order optimality condition to problems with quadratic constraints and without constant rank of the jacobian matrix.

In this paper we consider the minimization of a continuous function that is potentially not differentiable or not twice differentiable on the boundary of the feasible region. By exploiting an interior point technique, we present first- and second-order optimality conditions for this problem that reduces to classical ones when the derivative on the boundary is available. For this type of problems, existing necessary conditions often rely on the notion of subdifferential or become non-trivially weaker than the KKT condition in the (twice-)differentiable counterpart problems. In contrast, this paper presents a new set of first- and second-order necessary conditions that are derived without the use of subdifferential and reduces to exactly the KKT condition when (twice-)differentiability holds. As a result, these conditions are stronger than some existing ones considered for the discussed minimization problem when only non-negativity constraints are present. To solve for these optimality conditions in the special but important case of linearly constrained problems, we present two novel interior trust-region point algorithms and show that their worst-case computational efficiency in achieving the potentially stronger optimality conditions match the best known complexity bounds. Since this work considers a more general problem than the literature, our results also indicate that best known complexity bounds hold for a wider class of nonlinear programming problems.