Search

The Frank–Wolfe (FW) method, which implements efficient linear oracles that minimize linear approximations of the objective function over a fixed compact convex set, has recently received much attention in the optimization and machine learning literature. In this paper, we propose a new FW-type method for minimizing a smooth function over a compact set defined as the level set of a single difference-of-convex function, based on new generalized linear-optimization oracles (LO). We show that these LOs can be computed efficiently with closed-form solutions in some important optimization models that arise in compressed sensing and machine learning. In addition, under a mild strict feasibility condition, we establish the subsequential convergence of our nonconvex FW-type method. Since the feasible region of our generalized LO typically changes from iteration to iteration, our convergence analysis is completely different from those existing works in the literature on FW-type methods that deal with fixed feasible regions among subproblems. Finally, motivated by the away steps for accelerating FW-type methods for convex problems, we further design an away-step oracle to supplement our nonconvex FW-type method, and establish subsequential convergence of this variant. Numerical results on the matrix completion problem with standard datasets are presented to demonstrate the efficiency of the proposed FW-type method and its away-step variant. [ABSTRACT FROM AUTHOR]

Optimization problems involving minimization of a rank-one convex function over constraints modeling restrictions on the support of the decision variables emerge in various machine learning applications. These problems are often modeled with indicator variables for identifying the support of the continuous variables. In this paper we investigate compact extended formulations for such problems through perspective reformulation techniques. In contrast to the majority of previous work that relies on support function arguments and disjunctive programming techniques to provide convex hull results, we propose a constructive approach that exploits a hidden conic structure induced by perspective functions. To this end, we first establish a convex hull result for a general conic mixed-binary set in which each conic constraint involves a linear function of independent continuous variables and a set of binary variables. We then demonstrate that extended representations of sets associated with epigraphs of rank-one convex functions over constraints modeling indicator relations naturally admit such a conic representation. This enables us to systematically give perspective formulations for the convex hull descriptions of these sets with nonlinear separable or non-separable objective functions, sign constraints on continuous variables, and combinatorial constraints on indicator variables. We illustrate the efficacy of our results on sparse nonnegative logistic regression problems. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel projection method for finding a solution of equilibrium problems EP(C, f) in Euclidean spaces in which the bifunction does not require to be satisfied any monotonicity. Unlike methods for solving such a problem used in the literature, shrinking projection methods which may raise cost of computation, we suggest using an adaptive projection method incorporate with linesearch procedures to solve these problems when the bifunction does not satisfy the Lipschitz-type condition. These linesearches are unnecessary when f satisfies the Lipschitz-type condition with constants L 1 and L 2 . In case these constants are unknown, we propose to use the adaptive step sizes. All algorithms are proven to converge to a solution of the considered problem, and some numerical examples are also reported to illustrate the efficiency of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Natural policy gradient (NPG) and its variants are widely-used policy search methods in reinforcement learning. Inspired by prior work, a new NPG variant coined NPG-HM is developed in this paper, which utilizes the Hessian-aided momentum technique for variance reduction, while the sub-problem is solved via the stochastic gradient descent method. It is shown that NPG-HM can achieve the global last iterate ε -optimality with a sample complexity of O (ε - 2) , which is the best known result for natural policy gradient type methods under the generic Fisher non-degenerate policy parameterizations. The convergence analysis is built upon a relaxed weak gradient dominance property tailored for NPG under the compatible function approximation framework, as well as a neat way to decompose the error when handling the sub-problem. Moreover, numerical experiments on Mujoco-based environments demonstrate the superior performance of NPG-HM over other state-of-the-art policy gradient methods. [ABSTRACT FROM AUTHOR]

In this paper, a subgradient projection method for quasiconvex minimization problems is provided. By employing strong subdifferentials, it is proved that the generated sequence of the proposed algorithm converges to the solution of the minimization problem of a proper, lower semicontinuous, and strongly quasiconvex function (in the sense of Polyak in Soviet Math 7:72–75, 1966), under the same assumptions as those required for convex functions with the convex subdifferentials. Furthermore, a quasi-linear convergence rate of the iterates, extending similar results for the general quasiconvex case, is also provided. [ABSTRACT FROM AUTHOR]

Manifold optimization has recently gained significant attention due to its wide range of applications in various areas. This paper introduces the first Riemannian trust region method for minimizing an SC 1 function, which is a differentiable function that has a semismooth gradient vector field, on manifolds with convergence guarantee. We provide proof of both global and local convergence results, along with demonstrating the local superlinear convergence rate of our proposed method. As an application and to demonstrate our motivation, we utilize our trust region method as a subproblem solver within an augmented Lagrangian method for minimizing nonsmooth nonconvex functions over manifolds. This represents the first approach that fully explores the second-order information of the subproblem in the context of augmented Lagrangian methods on manifolds. Numerical experiments confirm that our method outperforms existing methods. [ABSTRACT FROM AUTHOR]

The Griewank test function for global unconstrained optimization has multiple local minima clustered around the global minimum at the origin. A new version of this test function is proposed that has a similar structure, but whose behavior at the local minima and maxima is non-smooth. This piecewise smooth version of the Griewank function represents an abs-factorable test case of objective functions for global non-smooth optimization as, for example, observed in the training of neural networks. [ABSTRACT FROM AUTHOR]

We translate a multiobjective optimization problem into a single objective Lipschitz problem by using the hypervolume criterion of the Pareto set. Deterministic global optimization methods allow one to track the whole Pareto front rather than converging to a single non-dominated solution. We augmented an efficient hypervolume computation technique with hypervolume increment strategy, and established Lipschitzianity of the resulting objective function. We used two deterministic Lipschitz optimization methods together with the hypervolume objective and benchmarked them against some state-of-the-art alternative multiobjective optimization methods, establishing the competitiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

In this paper, we study the finite-time convergence of the time-varying dynamical systems for solving convex and nonconvex optimization problems in different scenarios. We first show the asymptotic convergence of the trajectories of dynamical systems while only requiring convexity of the objective function. Under the Kurdyka–Łojasiewicz (KL) exponent of the objective function, we establish the finite-time convergence of the trajectories to the optima from any initial point. Making use of the Moreau envelope, we adapt our finite-time convergent algorithm to solve weakly convex nonsmooth optimization problems. In addition, we unify and extend the contemporary results on the KL exponent of the Moreau envelope of weakly convex functions. A dynamical system is also introduced to find a fixed point of a nonexpansive operator in finite time and fixed time under additional regularity properties. We then apply it to address the composite optimization problems with finite-time and fixed-time convergence. [ABSTRACT FROM AUTHOR]

In this paper, we develop an algorithm for federated principal component analysis (PCA) with emphases on both communication efficiency and data privacy. Generally speaking, federated PCA algorithms based on direct adaptations of classic iterative methods, such as simultaneous subspace iterations, are unable to preserve data privacy, while algorithms based on variable-splitting and consensus-seeking, such as alternating direction methods of multipliers (ADMM), lack in communication-efficiency. In this work, we propose a novel consensus-seeking formulation by equalizing subspaces spanned by splitting variables instead of equalizing variables themselves, thus greatly relaxing feasibility restrictions and allowing much faster convergence. Then we develop an ADMM-like algorithm with several special features to make it practically efficient, including a low-rank multiplier formula and techniques for treating subproblems. We establish that the proposed algorithm can better protect data privacy than classic methods adapted to the federated PCA setting. We derive convergence results, including a worst-case complexity estimate, for the proposed ADMM-like algorithm in the presence of the nonlinear equality constraints. Extensive empirical results are presented to show that the new algorithm, while enhancing data privacy, requires far fewer rounds of communication than existing peer algorithms for federated PCA. [ABSTRACT FROM AUTHOR]

In this paper, we investigate bounds of solution set of the polynomial complementarity problem. When a polynomial complementarity problem has a solution, we propose a lower bound of solution norm by entries of coefficient tensors of the polynomial. We prove that the proposing lower bound is larger than some existing lower bounds appeared in tensor complementarity problems and polynomial complementarity problems. When the solution set of a polynomial complementarity problem is nonempty, and the coefficient tensor of the leading term of the polynomial is an R 0 -tensor, we propose a new upper bound of solution norm of the polynomial complementarity problem by a quantity defining by an optimization problem. Furthermore, we prove that when coefficient tensors of the polynomial are partially symmetric, the proposing lower bound formula with respect to tensor tuples reaches the maximum value, and the proposing upper bound formula with respect to tensor tuples reaches the minimum value. Finally, by using such partial symmetry, we obtain bounds of solution norm by coefficients of the polynomial. [ABSTRACT FROM AUTHOR]

The sharp local minimality of feasible points of nonlinear optimization problems is known to possess a characterization by a strengthened version of the Karush–Kuhn–Tucker conditions, as long as the Mangasarian–Fromovitz constraint qualification holds. This strengthened condition is not easy to check algorithmically since it involves the topological interior of some set. In this article, we derive an algorithmically tractable version of this condition, called strong Karush–Kuhn–Tucker condition. We show that the Guignard constraint qualification is the weakest condition under which a feasible point is a strong Karush–Kuhn–Tucker point for every continuously differentiable objective function possessing the point as a sharp local minimizer. As an application, our results yield an algebraic characterization of strict local minimizers of linear programs with cardinality constraints. [ABSTRACT FROM AUTHOR]

The main goal of this paper is to relate the topologically relevant stationary points of a cardinality-constrained optimization problem and its continuous reformulation up to their type. For that, we focus on the non-degenerate M- and T-stationary points, respectively. Their M-index and T-index, respectively, which uniquely determine the global and local structure of optimization problems under consideration in algebraic terms, are traced. As novelty, we suggest to regularize the continuous reformulation for this purpose. The main consequence of our analysis is that the number of saddle points of the regularized continuous reformulation grows exponentially as compared to that of the initial cardinality-constrained optimization problem. This growth appears to be inherent and is reproduced in other relaxation attempts. [ABSTRACT FROM AUTHOR]

Stochastic gradient descent method and its variants constitute the core optimization algorithms that achieve good convergence rates for solving machine learning problems. These rates are obtained especially when these algorithms are fine-tuned for the application at hand. Although this tuning process can require large computational costs, recent work has shown that these costs can be reduced by line search methods that iteratively adjust the step length. We propose an alternative approach to stochastic line search by using a new algorithm based on forward step model building. This model building step incorporates second-order information that allows adjusting not only the step length but also the search direction. Noting that deep learning model parameters come in groups (layers of tensors), our method builds its model and calculates a new step for each parameter group. This novel diagonalization approach makes the selected step lengths adaptive. We provide convergence rate analysis, and experimentally show that the proposed algorithm achieves faster convergence and better generalization in well-known test problems. More precisely, SMB requires less tuning, and shows comparable performance to other adaptive methods. [ABSTRACT FROM AUTHOR]

In locally convex spaces, we introduce the new notion of approximate weakly efficient solution of the set-valued optimization problem with variable ordering structures (in short, SVOPVOS) and compare it with other kinds of solutions. Under the assumption of near D (·) -subconvexlikeness, we establish linear scalarization theorems of (SVOPVOS) in the sense of approximate weak efficiency. Finally, without any convexity, we obtain a nonlinear scalarization theorem of (SVOPVOS). We also present some examples to illustrate our results. [ABSTRACT FROM AUTHOR]

This study investigates the Poissonian image restoration problems. In particular, we propose a novel model that incorporates L 1 / L 2 minimization on the gradient as a regularization term combined with a box constraint and a nonlinear data fidelity term, specifically crafted to address the challenges caused by Poisson noise. We employ a splitting strategy, followed by the alternating direction method of multipliers (ADMM) to find a model solution. Furthermore, we show that under mild conditions, the sequence generated by ADMM has a sub-sequence that converges to a stationary point of the proposed model. Through numerical experiments on image deconvolution, super-resolution, and magnetic resonance imaging (MRI) reconstruction, we demonstrate superior performance made by the proposed approach over some existing gradient-based methods. [ABSTRACT FROM AUTHOR]

We propose an Efficient Inexact Learned Descent-type Algorithm (ELDA) for a class of nonconvex and nonsmooth variational models, where the regularization consists of a sparsity enhancing term and non-local smoothing term for learned features. The ELDA improves the performance of the LDA in Chen et al. (SIAM J Imag Sci 14(4), 1532–1564, 2021) by reducing the number of the subproblems from two to one for most of the iterations and allowing inexact gradient computation. We generate a deep neural network, whose architecture follows the algorithm exactly for low-dose CT (LDCT) reconstruction. The network inherits the convergence behavior of the algorithm and is interpretable as a solution of the varational model and parameter efficient. The experimental results from the ablation study and comparisons with several state-of-the-art deep learning approaches indicate the promising performance of the proposed method in solution accuracy and parameter efficiency. [ABSTRACT FROM AUTHOR]

The aim of this paper is to present dual optimality conditions for the difference of two extended real valued increasing co-radiant functions. We do this by first characterizing dual optimality conditions for the difference of two nonpositive increasing co-radiant functions. Finally, we present dual optimality conditions for the difference of two extended real valued increasing co-radiant functions. Our approach is based on the Toland–Singer formula. [ABSTRACT FROM AUTHOR]

We propose a second-order method for unconditional minimization of functions f(z) of complex arguments. We call it the mixed Newton method due to the use of the mixed Wirtinger derivative ∂ 2 f ∂ z ¯ ∂ z for computation of the search direction, as opposed to the full Hessian ∂ 2 f ∂ (z , z ¯) 2 in the classical Newton method. The method has been developed for specific applications in wireless network communications, but its global convergence properties are shown to be superior on a more general class of functions f, namely sums of squares of absolute values of holomorphic functions. In particular, for such objective functions minima are surrounded by attraction basins, while the iterates are repelled from other types of critical points. We provide formulas for the asymptotic convergence rate and show that in the scalar case the method reduces to the well-known complex Newton method for the search of zeros of holomorphic functions. In this case, it exhibits generically fractal global convergence patterns. [ABSTRACT FROM AUTHOR]

The following article presents and corroborates an outcome space branch-and-bound algorithm for solving the general linear multiplicative programming problem (GLMPP). In this new algorithm, GLMPP is transformed into its equivalent problem by introducing auxiliary variables. To compute the tight upper bound for the optimal value of GLMPP, the linear relaxation programming problem is assembled by using bound and hull linear approximations. Furthermore, the global convergence and computational complexity of the algorithm are demonstrated. Finally, the soundness and advantage of the proposed algorithm are validated by solving a demonstrative linear multiplicative problem. [ABSTRACT FROM AUTHOR]

Trends over the past two decades indicate that much of the performance gains of commercial optimization solvers is due to improvements in x86 hardware. To continue making progress, it is critical to consider alternative/specialized massively parallel computing architectures. In this work, we detail the development of an open-source source code transformation approach built using Symbolics.jl to construct McCormick-based relaxations of functions that enables their effective parallelized evaluation. We then apply this approach in a novel parallelized branch-and-bound routine that offloads lower- and upper-bounding problems to a GPU. The effectiveness of this new approach is demonstrated on three nonconvex problems of interest, where it yields convergence time improvements of 22–118x compared to an equivalent serial CPU implementation and in two cases outperforms vanilla branch-and-bound versions of existing state-of-the-art solvers that use tighter bounding techniques. This work exemplifies how deterministic global optimizers using alternative hardware architectures can compete with—or eventually outclass—even the most powerful serial CPU implementations, and to the best of the authors' knowledge, represents the first successful demonstration of deterministic global optimization using a GPU. [ABSTRACT FROM AUTHOR]

This paper studies the convergence of the proximal point method for quasiconvex functions in finite dimensional complete Riemannian manifolds. We prove initially that, in the general case, when the objective function is proper and lower semicontinuous, each accumulation point of the sequence generated by the method, if it exists, is a limiting critical point of the function. Then, under the assumptions that the sectional curvature of the manifold is bounded above by some non negative constant and the objective function is quasiconvex we analyze two cases. When the constant is zero, the global convergence of the algorithm to a limiting critical point is assured and if it is positive, we prove the local convergence for a class of quasiconvex functions, which includes Lipschitz functions. [ABSTRACT FROM AUTHOR]

In this study, we investigate a class of linear multiplicative programs with positive exponents. By introducing p additional variables, the original problem is reformulated into an equivalent problem (EP) within the output space. Subsequently, a novel global optimization algorithm is introduced to tackle EP. The algorithm primarily leverages two key techniques. One is the outer approximation technique, which tightens the relaxed feasible region of EP and improves the upper bound by carefully examining suitable feasible points. The other is the branch and bound technique to guarantee the global optimality of the solution. Numerical results confirm the effectiveness and practicality of the proposed algorithm, thereby underscoring its potential for real-world applications. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a modified Tseng's extragradient method with a new step-length rule to solve pseudo-monotone variational inequalities in real Hilbert spaces. Under suitable conditions, the sequence generated by this algorithm strongly converges to the common elements of the solution set of pseudo-monotone variational inequality problems and the fixed point set of k-demicontractive mappings. Finally, we give some numerical experiments to illustrate the effectiveness of the proposed algorithm. The main results of this paper generalize and improve some known results in the literature. [ABSTRACT FROM AUTHOR]

In this paper, for solving a broad class of large-scale nonconvex and nonsmooth optimization problems, we propose a stochastic two-step inertial Bregman proximal alternating linearized minimization (STiBPALM) algorithm with variance-reduced stochastic gradient estimators. And we show that SAGA and SARAH are variance-reduced gradient estimators. Under expectation conditions with the Kurdyka–Łojasiewicz property and some suitable conditions on the parameters, we obtain that the sequence generated by the proposed algorithm converges to a critical point. And the general convergence rate is also provided. Numerical experiments on sparse nonnegative matrix factorization and blind image-deblurring are presented to demonstrate the performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Tensor spectral p -norms are generalizations of matrix induced norms. Matrix induced norms are an important type of matrix norms, and tensor spectral p -norms are also important in applications. We discuss some basic properties of tensor spectral p -norms. We extend the submultiplicativity of the matrix spectral 2-norm to the tensor case, based on which we give a bound of the tensor spectral 2-norm and provide a fast method for computing spectral 2-norms of sum-of-squares tensors. To compute tensor spectral p -norms, we propose a higher order power method. Experiments show the high efficiency of the proposed methods and numerical results on spectral p -norms of random tensors are also given. [ABSTRACT FROM AUTHOR]

In this paper, we consider large-scale ranking problems where one is given a set of (possibly non-redundant) pairwise comparisons and the underlying ranking explained by those comparisons is desired. We show that stochastic gradient descent approaches can be leveraged to offer convergence to a solution that reveals the underlying ranking while requiring low-memory operations. We introduce several variations of this approach that offer a tradeoff in speed and convergence when the pairwise comparisons are noisy (i.e., some comparisons do not respect the underlying ranking). We prove theoretical results for convergence almost surely and study several regimes including those with full observations, partial observations, and noisy observations. Our empirical results give insights into the number of observations required as well as how much noise in those measurements can be tolerated. [ABSTRACT FROM AUTHOR]

We consider first-order methods with constant step size for minimizing locally Lipschitz coercive functions that are tame in an o-minimal structure on the real field. We prove that if the method is approximated by subgradient trajectories, then the iterates eventually remain in a neighborhood of a connected component of the set of critical points. Under suitable method-dependent regularity assumptions, this result applies to the subgradient method with momentum, the stochastic subgradient method with random reshuffling and momentum, and the random-permutations cyclic coordinate descent method. [ABSTRACT FROM AUTHOR]

We study a new variant of policy gradient method, named homotopic policy mirror descent (HPMD), for solving discounted, infinite horizon MDPs with finite state and action spaces. HPMD performs a mirror descent type policy update with an additional diminishing regularization term, and possesses several computational properties that seem to be new in the literature. When instantiated with Kullback–Leibler divergence, we establish the global linear convergence of HPMD applied to any MDP instance, for both the optimality gap, and a weighted distance to the set of optimal policies. We then unveil a phase transition, where both quantities exhibit local acceleration, and converge at a superlinear rate after the optimality gap falls below certain instance-dependent threshold. With local acceleration and diminishing regularization, we establish the first result among policy gradient methods on certifying and characterizing the limiting policy, by showing, with a non-asymptotic characterization, that the last-iterate policy converges to the unique optimal policy with the maximal entropy. We then extend all the aforementioned results to HPMD instantiated with a broad class of decomposable Bregman divergences, demonstrating the generality of the these computational properties. As a byproduct, we discover the finite-time exact convergence for some commonly used Bregman divergences, implying the continuing convergence of HPMD to the limiting policy even if the current policy is already optimal. Finally, we develop a stochastic version of HPMD and establish similar convergence properties. By exploiting the local acceleration, we show that when a generative model is available for policy evaluation, with a small enough ϵ 0 , for any target precision ϵ ≤ ϵ 0 , an ϵ -optimal policy can be learned with O ~ (| S | | A | / ϵ 0 2) samples with probability 1 - O (ϵ 0 1 / 3 ) . [ABSTRACT FROM AUTHOR]

We consider the multilinear polytope defined as the convex hull of the set of binary points z, satisfying a collection of equations of the form z e = ∏ v ∈ e z v for all e ∈ E . The complexity of the facial structure of the multilinear polytope is closely related to the acyclicity degree of the underlying hypergraph. We obtain a polynomial-size extended formulation for the multilinear polytope of β -acyclic hypergraphs, hence characterizing the acyclic hypergraphs for which such a formulation can be constructed. [ABSTRACT FROM AUTHOR]

In this work, we develop a new complexity metric for an important class of low-rank matrix optimization problems in both symmetric and asymmetric cases, where the metric aims to quantify the complexity of the nonconvex optimization landscape of each problem and the success of local search methods in solving the problem. The existing literature has focused on two recovery guarantees. The RIP constant is commonly used to characterize the complexity of matrix sensing problems. On the other hand, the incoherence and the sampling rate are used when analyzing matrix completion problems. The proposed complexity metric has the potential to generalize these two notions and also applies to a much larger class of problems. To mathematically study the properties of this metric, we focus on the rank-1 generalized matrix completion problem and illustrate the usefulness of the new complexity metric on three types of instances, namely, instances with the RIP condition, instances obeying the Bernoulli sampling model, and a synthetic example. We show that the complexity metric exhibits a consistent behavior in the three cases, even when other existing conditions fail to provide theoretical guarantees. These observations provide a strong implication that the new complexity metric has the potential to generalize various conditions of optimization complexity proposed for different applications. Furthermore, we establish theoretical results to provide sufficient conditions and necessary conditions for the existence of spurious solutions in terms of the proposed complexity metric. This contrasts with the RIP and incoherence conditions that fail to provide any necessary condition. [ABSTRACT FROM AUTHOR]

This article presents a self-adjustable branch-and-bound algorithm for globally solving a class of linear multiplicative programming problems (LMP). In this algorithm, a self-adjustable branching rule is introduced and it can continuously update the upper bound for the optimal value of LMP by selecting suitable branching point under certain conditions, which differs from the standard bisection rule. The proposed algorithm further integrates the linear relaxation program and the self-adjustable branching rule. The dependability and robustness of the proposed algorithm are demonstrated by establishing the global convergence. Furthermore, the computational complexity of the proposed algorithm is estimated. Finally, numerical results validate the effectiveness of the self-adjustable branching rule and demonstrate the feasibility of the proposed algorithm. [ABSTRACT FROM AUTHOR]

We study T-semidefinite programming (SDP) relaxation for constrained polynomial optimization problems (POPs). T-SDP relaxation for unconstrained POPs was introduced by Zheng et al. (JGO 84:415–440, 2022). In this work, we propose a T-SDP relaxation for POPs with polynomial inequality constraints and show that the resulting T-SDP relaxation formulated with third-order tensors can be transformed into the standard SDP relaxation with block-diagonal structures. The convergence of the T-SDP relaxation to the optimal value of a given constrained POP is established under moderate assumptions as the relaxation level increases. Additionally, the feasibility and optimality of the T-SDP relaxation are discussed. Numerical results illustrate that the proposed T-SDP relaxation enhances numerical efficiency. [ABSTRACT FROM AUTHOR]

First-order optimization methods have long been established as effective methods for solving large-scale unconstrained optimization problems. Over time, these methods have been extended to address the task of finding zeros of single-valued monotone maps within the framework of finite-dimensional Hilbert spaces. Notably, Abubakar et al. (Comput Appl Math 39(2):129, 2020) recently introduced a projection method that incorporates a convex combination of two distinct positive spectral coefficients to find the zeros of a single-valued monotone map. Motivated by their work, this paper generalises the Abubakar et al. technique to a more general setting of infinite-dimensional Hilbert spaces. Under mild assumptions, we establish weak convergence of the generated sequence of iterates. Furthermore, the extended method does not involve computing the resolvent of the monotone operator. [ABSTRACT FROM AUTHOR]

We investigate how to solve smooth matrix optimization problems with general linear inequality constraints on the eigenvalues of a symmetric matrix. We present solution methods to obtain exact global minima for linear objective functions, i.e., F (X) = ⟨ C , X ⟩ , and perform exact projections onto the eigenvalue constraint set. Two first-order algorithms are developed to obtain first-order stationary points for general non-convex objective functions. Both methods are proven to converge sublinearly when the constraint set is convex. Numerical experiments demonstrate the applicability of both the model and the methods. [ABSTRACT FROM AUTHOR]

We study several discretizations of a second order gradient-like system with damping. We first consider an explicit scheme with a linear damping in finite dimension. We prove that every solution converges if the nonlinearity satisfies a global Lojasiewicz inequality. Convergence rates are also established. In the case of a strong nonlinear damping, we prove convergence of every solution for a fully implicit scheme in the one-dimensional case, even if the nonlinearity does not satisfy a Lojasiewicz inequality. The optimality of the damping is also established. Numerical simulations illustrate the theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, we investigate sequential M-stationarity conditions for a class of nonsmooth nonconvex general optimization problems. We introduce various types of such conditions and compare them with previously established conditions in smooth or convex cases. The application of the derived results is demonstrated in the context of nonsmooth sparsity-constrained optimization problems. Additionally, we devise a Lagrangian-type algorithm for a specific case of smooth sparsity problems. Several examples are presented throughout the paper to illustrate the results. [ABSTRACT FROM AUTHOR]

This paper investigates sparse optimization problems characterized by a sparse group structure, where element- and group-level sparsity are jointly taken into account. This particular optimization model has exhibited notable efficacy in tasks such as feature selection, parameter estimation, and the advancement of model interpretability. Central to our study is the scrutiny of the ℓ 0 and ℓ 2 , 0 norm regularization model, which, in comparison to alternative surrogate formulations, presents formidable computational challenges. We embark on our study by conducting the analysis of the optimality conditions of the sparse group optimization problem, leveraging the notion of a γ -stationary point, whose linkage to local and global minimizer is established. In a subsequent facet of our study, we develop a novel subspace Newton algorithm for sparse group ℓ 0 optimization problem and prove its global convergence property as well as local second-order convergence rate. Experimental results reveal the superlative performance of our algorithm in terms of both precision and computational expediency, thereby outperforming several state-of-the-art solvers. [ABSTRACT FROM AUTHOR]