Search

The optimization problems with a sparsity constraint is a class of important global optimization problems. A typical type of thresholding algorithms for solving such a problem adopts the traditional full steepest descent direction or Newton-like direction as a search direction to generate an iterate on which a certain thresholding is performed. Traditional hard thresholding discards a large part of a vector, and thus some important information contained in a dense vector has been lost in such a thresholding process. Recent study (Zhao in SIAM J Optim 30(1): 31–55, 2020) shows that the hard thresholding should be applied to a compressible vector instead of a dense vector to avoid a big loss of information. On the other hand, the optimal k-thresholding as a novel thresholding technique may overcome the intrinsic drawback of hard thresholding, and performs thresholding and objective function minimization simultaneously. This motivates us to propose the so-called partial gradient optimal thresholding (PGOT) method and its relaxed versions in this paper. The PGOT is an integration of the partial gradient and the optimal k-thresholding technique. The solution error bound and convergence for the proposed algorithms have been established in this paper under suitable conditions. Application of our results to the sparse optimization problems arising from signal recovery is also discussed. Experiment results from synthetic data indicate that the proposed algorithm is efficient and comparable to several existing algorithms.

The optimization problem with sparsity arises in many areas of science and engineering such as compressed sensing, image processing, statistical learning and data sparse approximation. In this paper, we study the dual-density-based reweighted $$\ell _{1}$$ ℓ 1 -algorithms for a class of $$\ell _{0}$$ ℓ 0 -minimization models which can be used to model a wide range of practical problems. This class of algorithms is based on certain convex relaxations of the reformulation of the underlying $$\ell _{0}$$ ℓ 0 -minimization model. Such a reformulation is a special bilevel optimization problem which, in theory, is equivalent to the underlying $$\ell _{0}$$ ℓ 0 -minimization problem under the assumption of strict complementarity. Some basic properties of these algorithms are discussed, and numerical experiments have been carried out to demonstrate the efficiency of the proposed algorithms. Comparison of numerical performances of the proposed methods and the classic reweighted $$\ell _1$$ ℓ 1 -algorithms has also been made in this paper.

The sparse optimization problems arise in many areas of science and engineering, such as compressed sensing, image processing, statistical and machine learning. The l 0 -minimization problem is one...

In this paper, we study the generalized Hadamard well-posedness of infinite vector optimization problems (IVOP). Without the assumption of continuity with respect to the first variable, the upper semicontinuity and closedness of constraint set mappings are established. Under weaker assumptions, sufficient conditions of generalized Hadamard well-posedness for IVOP are obtained under perturbations of both the objective function and the constraint set. We apply our results to the semi-infinite vector optimization problem and the semi-infinite multi-objective optimization problem.

We consider a class of nonsmooth convex optimization problems where the objective function is a convex differentiable function regularized by the sum of the group reproducing kernel norm and $$\ell _1$$l1-norm of the problem variables. This class of problems has many applications in variable selections such as the group LASSO and sparse group LASSO. In this paper, we propose a proximal Landweber Newton method for this class of convex optimization problems, and carry out the convergence and computational complexity analysis for this method. Theoretical analysis and numerical results show that the proposed algorithm is promising.

The uniqueness of sparsest solutions of underdetermined linear systems plays a fundamental role in the newly developed compressed sensing theory. Several new algebraic concepts, including the sub-mutual coherence, scaled mutual coherence, coherence rank, and sub-coherence rank, are introduced in this paper in order to develop new and improved sufficient conditions for the uniqueness of sparsest solutions. The coherence rank of a matrix with normalized columns is the maximum number of absolute entries in a row of its Gram matrix that are equal to the mutual coherence. The main result of this paper claims that when the coherence rank of a matrix is low, the mutual-coherence-based uniqueness conditions for the sparsest solution of a linear system can be improved. Furthermore, we prove that the Babel-function-based uniqueness can be also improved by the so-called sub-Babel function. Moreover, we show that the scaled-coherence-based uniqueness conditions can be developed, and that the right-hand-side vector $b$ of a linear system, the support overlap of solutions, the orthogonal matrix out of the singular value decomposition of a matrix, and the range property of a transposed matrix can be also integrated into the criteria for the uniqueness of the sparsest solution of an underdetermined linear system.

The problem of finding a rank-one solution to a system of linear matrix equations arises from many practical applications. Given a system of linear matrix equations, however, such a low-rank solution does not always exist. In this paper, we aim at developing some sufficient conditions for the existence of a rank-one solution to the system of homogeneous linear matrix equations (HLME) over the positive semidefinite cone. First, we prove that an existence condition of a rank-one solution can be established by a homotopy invariance theorem. The derived condition is closely related to the so-called P"@A property of the function defined by quadratic transformations. Second, we prove that the existence condition for a rank-one solution can be also established through the maximum rank of the (positive semidefinite) linear combination of given matrices. It is shown that an upper bound for the rank of the solution to a system of HLME over the positive semidefinite cone can be obtained efficiently by solving a semidefinite programming (SDP) problem. Moreover, a sufficient condition for the nonexistence of a rank-one solution to the system of HLME is also established in this paper.

Numerical experiments have indicated that the reweighted $\ell_1$-minimization performs exceptionally well in locating sparse solutions of underdetermined linear systems of equations. We show that reweighted $\ell_1$-methods are intrinsically associated with the minimization of the so-called merit functions for sparsity, which are essentially concave approximations to the cardinality function. Based on this observation, we further show that a family of reweighted $\ell_1$-algorithms can be systematically derived from the perspective of concave optimization through the linearization technique. In order to conduct a unified convergence analysis for this family of algorithms, we introduce the concept of the range space property (RSP) of a matrix and prove that if its adjoint has this property, the reweighted $\ell_1$-algorithm can find a sparse solution to the underdetermined linear system provided that the merit function for sparsity is properly chosen. In particular, some convergence conditions for the Can...

We consider spline interpolation problems where information about the approximated function is given by means of interval estimates for the function values over ranges of x-values instead of specific knots. We propose two robust univariate spline models formulated as convex semi-infinite optimization problems. We present simplified equivalent formulations of both models as finite explicit convex optimization problems for splines of degrees up to 3. This makes it possible to use existing convex optimization algorithms and software.

While the product of finitely many convex functions has been investigated in the field of global optimization, some fundamental issues such as the convexity condition and the Legendre-Fenchel transform for the product function remain unresolved. Focusing on quadratic forms, this paper is aimed at addressing the question: When is the product of finitely many positive definite quadratic forms convex, and what is the Legendre-Fenchel transform for it? First, we show that the convexity of the product is determined intrinsically by the condition number of so-called ‘scaled matrices’ associated with quadratic forms involved. The main result claims that if the condition number of these scaled matrices are bounded above by an explicit constant (which depends only on the number of quadratic forms involved), then the product function is convex. Second, we prove that the Legendre-Fenchel transform for the product of positive definite quadratic forms can be expressed, and the computation of the transform amounts to finding the solution to a system of equations (or equally, finding a Brouwer’s fixed point of a mapping) with a special structure. Thus, a broader question than the open “Question 11” in Hiriart-Urruty (SIAM Rev. 49, 225–273, 2007) is addressed in this paper.

Traditional univariate cubic spline models assume that the position and function value of each knot are given precisely. It has been observed that errors in data could result in significant fluctuations of the resulting spline. To handle situations that involve uncertainty only in measurements of function values, the concept of a robust spline has been developed in the literature. We propose a more general concept of a PH-robust cubic spline that takes into account also uncertainty in positions of measurements (knots or boundary points) using the paradigm of robust optimization. This bridges the robustness concepts developed in the interpolation/approximation and the optimization communities. Our model handles the case of "coordinated" variations of positions of measurements. It is formulated as a semi-infinite convex optimization problem. We develop a reformulation of the model as a finite explicit convex optimization problem, which makes it possible to use standard convex optimization algorithms for computation.

We establish two global bounds measuring the distance from any vector to the solution set of the co-coercive variational inequality. To prove our results, we use the fact that the co-coercivity condition is sufficient for the (strong) monotonicity of (perturbed) fixed point and normal maps associated with variational inequalities.

With the objective of generating "shape-preserving" smooth interpolating curves that represent data with abrupt changes in magnitude and/or knot spacing, we study a class of first-derivative-based $$\mathcal{C}^1$$ -smooth univariate cubic L 1 splines. An L 1 spline minimizes the L 1 norm of the difference between the first-order derivative of the spline and the local divided difference of the data. Calculating the coefficients of an L 1 spline is a nonsmooth non-linear convex program. Via Fenchel's conjugate transformation, the geometric dual program is a smooth convex program with a linear objective function and convex cubic constraints. The dual-to-primal transformation is accomplished by solving a linear program.

Homogeneous programming is an important class of optimization problems. The purpose of this note is to give a truly equivalent characterization of KKT points of homogeneous programming problems, correcting a result given by Lasserre and Hiriart-Urruty in Ref. 1.

Based on the recent theoretical results of Zhao and Li [Math. Oper. Res., 26 (2001), pp. 119--146], we present in this paper a new path-following method for nonlinear P* complementarity problems. Different from most existing interior-point algorithms that are based on the central path, this algorithm tracks the "regularized central path" which exists for any continuous P* problem. It turns out that the algorithm is globally convergent for any P* problem provided that its solution set is nonempty. By different choices of the parameters in the algorithm, the iterative sequence can approach to different types of points of the solution set. Moreover, local superlinear convergence of this algorithm can also be achieved under certain conditions.

The generalized mean function has been widely used in convex analysis and mathematical programming. This paper studies a further generalization of such a function. A necessary and sufficient condition is obtained for the convexity of a generalized function. Additional sufficient conditions that can be easily checked are derived for the purpose of identifying some classes of functions which guarantee the convexity of the generalized functions. We show that some new classes of convex functions with certain regularity (such as S*-regularity) can be used as building blocks to construct such generalized functions.

We study several properties of a homotopy solution path associated with continuous nonlinear quasi-monotone complementarity problems. We use Poincare-Bohn's homotopy invariance theorem of degree to derive an alternative theorem. Based on this result, a sufficient condition is established to assure the existence and boundedness of this homotopy solution path. Our results provide a theoretical basis to develop a new computational method for quasi-monotone complementarity problems.

Let K be a closed convex subset in Rn, and let f; g and h be three functions from Rn into itself. We consider the following nonlinear projection equation (NPE): h(x) = :K (g(x)− f(x)); x ∈ R; (1) where :K (·) is the orthogonal projection operator onto the set K . This equation provides a uni

Most known continuation methods for P0 complementarity problems require some restrictive assumptions, such as the strictly feasible condition and the properness condition, to guarantee the existence and the boundedness of certain homotopy continuation trajectory. To relax such restrictions, we propose a new homotopy formulation for the complementarity problem based on which a new homotopy continuation trajectory is generated. For P0 complementarity problems, the most promising feature of this trajectory is the assurance of the existence and the boundedness of the trajectory under a condition that is strictly weaker than the standard ones used widely in the literature of continuation methods. Particularly, the often-assumed strictly feasible condition is not required here. When applied to P* complementarity problems, the boundedness of the proposed trajectory turns out to be equivalent to the solvability of the problem, and the entire trajectory converges to the (unique) least element solution provided that it exists. Moreover, for monotone complementarity problems, the whole trajectory always converges to a least 2-norm solution provided that the solution set of the problem is nonempty. The results presented in this paper can serve as a theoretical basis for constructing a new path-following algorithm for solving complementarity problems, even for the situations where the solution set is unbounded.

For P0-complementarity problems, most existing non--interior-point path-following methods require the existence of a strictly feasible point. (For a P*-complementarity problem, the existence of a strictly feasible point is equivalent to the nonemptyness and the boundedness of the solution set.) In this paper, we propose a new homotopy formulation for complementarity problems by which a new non--interior-point continuation trajectory is generated. The existence and the boundedness of this non--interior-point trajectory for P0-complementarity problems are proved under a very mild condition that is weaker than most conditions used in the literature. One prominent feature of this condition is that it may hold even when the often-assumed strict feasibility condition fails to hold. In particular, for a P*-problem it turns out that the new non--interior-point trajectory exists and is bounded if and only if the problem has a solution. We also study the convergence of this trajectory and characterize its limiting point as the parameter approaches zero.

We prove sufficient conditions for the monotonicity and the strong monotonicity of fixed point and normal maps associated with variational inequality problems over a general closed convex set. Sufficient conditions for the strong monotonicity of their perturbed versions are also shown. These results include some well known in the literature as particular instances. Inspired by these results, we propose a modified Solodov and Svaiter iterative algorithm for the variational inequality problem whose fixed point map or normal map is monotone.

Strict feasibility plays an important role in the development of the theoryand algorithms of complementarity problems. In this paper, we establishsufficient conditions to ensure strict feasibility of a nonlinearcomplementarity problem. Our analysis method, based on a newly introducedconcept of μ-exceptional sequence, can be viewed as a unified approachfor proving the existence of a strictly feasible point. Some equivalentconditions of strict feasibility are also developed for certaincomplementarity problems. In particular, we show that aP*-complementarity problem is strictly feasible if and only ifits solution set is nonempty and bounded.

Quasi-P*-maps and P(τ, α, β)-maps defined in this paper are two large classes of nonlinear mappings which are broad enough to include P*-maps as special cases. It is of interest that the class of quasi-P*-maps also encompasses quasimonotone maps (in particular, pseudomonotone maps) as special cases. Under a strict feasibility condition, it is shown that the nonlinear complementarity problem has a solution if the function is a nonlinear quasi-P*-map or P(τ, α, β)-map. This result generalizes a classical Karamardian existence theorem and a recent result concerning quasimonotone maps established by Hadjisawas and Schaible, but restricted to complementarity problems. A new existence result under an exceptional regularity condition is also established. Our method is based on the concept of exceptional family of elements for a continuous function, which is a powerful tool for investigating the solvability of complementarity problems.

In this paper, an alternative theorem, and hence a sufficient solution condition, is established for generalized variational inequality problems. The concept of exceptional family for generalized variational inequality is introduced. This concept is general enough to include as special cases the notions of exceptional family of elements and the D-orientation sequence for continuous functions. Particularly, we apply the alternative theorem for investigating the solvability of the nonlinear complementarity problems with so-called quasi-P^M"*-maps, which are broad enough to encompass the quasi-monotone maps and P"*-maps as the special cases. An existence theorem for this class of complementarity problems is established, which significantly generalizes several previous existence results in the literature.

Using the homotopy invariance property of the degree and a newly introduced concept of the interior-point-$\varepsilon$-exceptional family for continuous functions, we prove an alternative theorem concerning the existence of a certain interior-point of a continuous complementarity problem. Based on this result, we develop several sufficient conditions to assure some desirable properties (nonemptyness, boundedness, and upper-semicontinuity) of a multivalued mapping associated with continuous (nonmonotone) complementarity problems corresponding to semimonotone, P$(\tau, \alpha, \beta)$-, quasi-P*-, and exceptionally regular maps. The results proved in this paper generalize well-known results on the existence of central paths in continuous P0 complementarity problems.

We introduce the new concept of d-orientation sequence for continuous functions. It is shown that if there does not exist a d-orientation sequence for a continuous function, then the corresponding complementarity problem (CP) has a solution. We believe that such a result characterizes an intrinsic property of CPs. As the concept of ''exceptional family of elements'', the notion of ''d-orientation sequence of a function'' is also a powerful tool for investigating the existence theorems of CPs. We use this new tool to establish, among other things, a new existence result for a class of P"*-mapping CPs.

We establish several new existence theorems for nonlinear variational inequalities with generalized positively homogeneous functions. The results presented here are general enough to include two More existence theorems of complementarity problems as special cases. We also establish an existence result for nonlinear complementarity problems with an exceptional regularity map. The concept of an exceptional family for a variational inequality plays a key role in our analysis.

Two interior-point algorithms using a wide neighborhood of the central path are proposed to solve nonlinear P*-complementarity problems. The proof of the polynomial complexity of the first method requires the problem to satisfy a scaled Lipschitz condition. When specialized to monotone complementarity problems, the results of the first method are similar to those in Ref. 1. The second method is quite different from the first in that the global convergence proof does not require the scaled Lipschitz assumption. However, at each step of this algorithm, one has to compute an approximate solution of a nonlinear system such that a certain accuracy requirement is satisfied.

This paper introduces the concept of exceptional family for nonlinear variational inequality problems. Among other things, we show that the nonexistence of an exceptional family is a sufficient condition for the existence of a solution to variational inequalities. This sufficient condition is weaker than many known solution conditions and it is also necessary for pseudomonotone variational inequalities. From the results in this paper, we believe that the concept of exceptional families of variational inequalities provides a new powerful tool for the study of the existence theory for variational inequalities.

In this paper, we prove that each monotone variational inequality is equivalent to a two-mapping variational inequality problem. On the basis of this fact, a new class of iterative methods for the solution of nonlinear monotone variational inequality problems is presented. The global convergence of the proposed methods is established under the monotonicity assumption. The conditions concerning the implementability of the algorithms are also discussed. The proposed methods have a close relationship to the Douglas–Rachford operator splitting method for monotone variational inequalities.

This paper introduces a new concept of exceptional family of elements (abbreviated, exceptional family) for a finite-dimensional nonlinear variational inequality problem. By using this new concept, we establish a general sufficient condition for the existence of a solution to the problem. Such a condition is used to develop several new existence theorems. Among other things, a sufficient and necessary condition for the solvability of pseudo-monotone variational inequality problem is proved. The notion of coercivity of a function and related classical existence theorems for variational inequality are also generalized. Finally, a solution condition for a class of nonlinear complementarity problems with so-called P_* -mappings is also obtained.

A new concept of exceptional family for nonlinear variational inequalities over polyhedral sets is introduced in this paper. It generalizes the concepts for complementarity problem introduced by Smith and Isac. We applied the new analytical tool to the study of existence problem for variational inequalities. It is shown that our existence condition is weaker than most of the sufficient conditions which have been known.

A new class of iterative methods are presented for monotone generalized variational inequality problems. These methods, which base on an equivalent formulation of the original problem, can be viewed as the extension of the symmetric projection rnethod for monotone variational inequalities. The global convergence of the methods is estab-lished under the monotonicity assumption on the functions associated the problem.Specialization of the proposed algorithms and related results to several special cases are also discussed. Moreover, two combination methods are presented for affine monotone problems. and their global and Q-linear convergence are also established

The Kantorovich function $(x^TAx)(x^T A^{-1} x)$, where $A$ is a positive definite matrix, is not convex in general. From matrix/convex analysis point of view, it is interesting to address the question: When is this function convex? In this paper, we investigate the convexity of this function by the condition number of its matrix. In 2-dimensional space, we prove that the Kantorovich function is convex if and only if the condition number of its matrix is bounded above by $3+2\sqrt{2}, $ and thus the convexity of the function with two variables can be completely characterized by the condition number. The upper bound `$3+2\sqrt{2} $' is turned out to be a necessary condition for the convexity of Kantorovich functions in any finite-dimensional spaces. We also point out that when the condition number of the matrix (which can be any dimensional) is less than or equal to $\sqrt{5+2\sqrt{6}}, $ the Kantorovich function is convex. Furthermore, we prove that this general sufficient convexity condition can be remarkably improved in 3-dimensional space. Our analysis shows that the convexity of the function is closely related to some modern optimization topics such as the semi-infinite linear matrix inequality or 'robust positive semi-definiteness' of symmetric matrices. In fact, our main result for 3-dimensional cases has been proved by finding an explicit solution range to some semi-infinite linear matrix inequalities., Comment: 24 pages

Matrix rank minimization problems are gaining plenty of recent attention in both mathematical and engineering fields. This class of problems, arising in various and across-discipline applications, is known to be NP-hard in general. In this paper, we aim at providing an approximation theory for the rank minimization problem, and prove that a rank minimization problem can be approximated to any level of accuracy via continuous optimization (especially, linear and nonlinear semidefinite programming) problems. One of the main results in this paper shows that if the feasible set of the problem has a minimum rank element with the least Frobenius norm, then any accumulation point of solutions to the approximation problem, as the approximation parameter tends to zero, is a minimum rank solution of the original problem. The tractability under certain conditions and convex relaxation of the approximation problem are also discussed. An immediate application of this theory to the system of quadratic equations is presented in this paper. It turns out that the condition for such a system without a nonzero solution can be characterized by a rank minimization problem, and thus the proposed approximation theory can be used to establish some sufficient conditions for the system to possess only zero solution.