Search

In this paper, we propose a hybrid inexact regularized Newton and negative curvature method for solving unconstrained nonconvex problems. The descent direction is chosen based on different conditions, either the negative curvature or the inexact regularized direction. In addition, to minimize computational costs while obtaining the negative curvature, we employ a dimensionality reduction strategy to verify if the Hessian matrix exhibits negative curvatures within a three-dimensional subspace. We show that the proposed method can achieve the best-known global iteration complexity if the Hessian of the objective function is Lipschitz continuous on a certain compact set. Two simplified methods for nonconvex and strongly convex problems are analyzed as specific instances of the proposed method. We show that under the local error bound assumption with respect to the gradient, the distance between iterations generated by our proposed method and the local solution set converges to 0 at a superlinear rate. Additionally, for strongly convex problems, the quadratic convergence rate can be achieved. Extensive numerical experiments show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

The generalized alternating direction method of multipliers (ADMM) of Xiao et al. (Math Prog Comput 10:533–555, 2018) aims at the two-block linearly constrained composite convex programming problem, in which each block is in the form of "nonsmooth + quadratic". However, in the case of non-quadratic (but smooth), this method may fail unless the favorable structure of "nonsmooth + smooth" is no longer used. This paper aims to remedy this defect by using a majorized technique to approximate the augmented Lagrangian function, so that the corresponding subproblems can be decomposed into some smaller problems and then solved separately. Furthermore, the recent symmetric Gauss–Seidel (sGS) decomposition theorem guarantees the equivalence between the bigger subproblem and these smaller ones. This paper focuses on convergence analysis, that is, we prove that the sequence generated by the proposed method converges globally to a Karush–Kuhn–Tucker point of the considered problem. Finally, we do some numerical experiments on a kind of simulated convex composite optimization problems which illustrate that the proposed method is evidently efficient. [ABSTRACT FROM AUTHOR]

Magnetic Resonance Imaging (MRI) is a kind of medical imaging technology used for diagnostic imaging of diseases, but its image quality may be suffered by the long acquisition time. The compressive sensing (CS) based strategy may decrease the reconstruction time greatly, but it needs efficient reconstruction algorithms to produce high-quality and reliable images. This paper focuses on the algorithmic improvement for the sparse reconstruction of CS-MRI, especially considering a non-smooth convex minimization problem which is composed of the sum of a total variation regularization term and a ℓ 1 -norm term of the wavelet transformation. The partly motivation of targeting the dual problem is that the dual variables are involved in relatively low-dimensional subspace. Instead of solving the primal model as usual, we turn our attention to its associated dual model composed of three variable blocks and two separable non-smooth function blocks. However, the directly extended alternating direction method of multipliers (ADMM) must be avoided because it may be divergent, although it usually performs well numerically. In order to solve the problem, we employ a symmetric Gauss–Seidel (sGS) technique based ADMM. Compared with the directly extended ADMM, this method only needs one additional iteration, but its convergence can be guaranteed theoretically. Besides, we also propose a generalized variant of ADMM because this method has been illustrated to be efficient for solving semidefinite programming in the past few years. Finally, we do extensive experiments on MRI reconstruction using some simulated and real MRI images under different sampling patterns and ratios. The numerical results demonstrate that the proposed algorithms significantly achieve high reconstruction accuracies with fast computational speed. [ABSTRACT FROM AUTHOR]

In this paper, we propose a generalized alternating direction method of multipliers (ADMM) with semi-proximal terms for solving a class of convex composite conic optimization problems, of which some are high-dimensional, to moderate accuracy. Our primary motivation is that this method, together with properly chosen semi-proximal terms, such as those generated by the recent advance of block symmetric Gauss-Seidel technique, is capable of tackling these problems. Moreover, the proposed method, which relaxes both the primal and the dual variables in a natural way with a common relaxation factor in the interval of (0, 2), has the potential of enhancing the performance of the classic ADMM. Extensive numerical experiments on various doubly non-negative semidefinite programming problems, with or without inequality constraints, are conducted. The corresponding results showed that all these multi-block problems can be successively solved, and the advantage of using the relaxation step is apparent. [ABSTRACT FROM AUTHOR]

Transform invariant low-rank textures, referred to as TILT, can accurately and robustly extract textural or geometric information in a 3D from user-specified windows in 2D in spite of significant corruptions and warping. It was discovered that the task can be characterized, both theoretically and numerically, by solving a sequence of matrix nuclear-norm and ℓ1-norm involved convex minimization problems. For solving this problem, the direct extension of alternating direction method of multipliers (ADMM) in an usual Gauss-Seidel manner often performs numerically well in practice but there is no theoretical guarantee on its convergence. In this paper, we resolve this dilemma by using the novel symmetric Gauss-Seidel (sGS)-based ADMM developed by Li et al. (Math Program 155:333-373, 2016). The sGS-ADMM is guaranteed to converge, and we shall demonstrate in this paper that it is also practically efficient than the directly extended ADMM. When the sGS technique is applied to this particular problem, we show that only one variable needs to be re-updated, and this updating hardly imposes any excessive computational cost. The sGS decomposition theorem of Li et al. (Math. Program., 2018. 10.1007/s10107-018-1247-7) establishes the equivalent between sGS-ADMM and the classical ADMM with an additional semi-proximal term, so the convergence result is followed directly. Extensive experiments illustrate that the sGS-ADMM and its generalized variant have superior numerical efficiency over the directly extended ADMM. [ABSTRACT FROM AUTHOR]

This study aims to present a limited memory BFGS algorithm to solve non-convex minimization problems, and then use it to find the largest eigenvalue of a real symmetric positive definite matrix. The proposed algorithm is based on the modified secant equation, which is used to the limited memory BFGS method without more storage or arithmetic operations. The proposed method uses an Armijo line search and converges to a critical point without convexity assumption on the objective function. More importantly, we do extensive experiments to compute the largest eigenvalue of the symmetric positive definite matrix of order up to 54,929 from the UF sparse matrix collection, and do performance comparisons with EIGS (a Matlab implementation for computing the first finite number of eigenvalues with largest magnitude). Although the proposed algorithm converges to a critical point, not a global minimum theoretically, the compared results demonstrate that it works well, and usually finds the largest eigenvalue of medium accuracy. [ABSTRACT FROM AUTHOR]

Nonlinear conjugate gradient method is very popular in solving large-scale unconstrained minimization problems due to its simple iterative form and lower storage requirement. In the recent years, it was successfully extended to solve higher-dimension monotone nonlinear equations. Nevertheless, the research activities on conjugate gradient method in symmetric equations are just beginning. This study aims to developing, analyzing, and validating a family of nonlinear conjugate gradient methods for symmetric equations. The proposed algorithms are based on the latest, and state-of-the-art descent conjugate gradient methods for unconstrained minimization. The series of proposed methods are derivative-free, where the Jacobian information is needless at the full iteration process. We prove that the proposed methods converge globally under some appropriate conditions. Numerical results with differentiable parameter's values and performance comparisons with another solver CGD to demonstrate the superiority and effectiveness of the proposed algorithms are reported. [ABSTRACT FROM AUTHOR]

Given a set of corrupted data drawn from a union of multiple subspace, the subspace recovery problem is to segment the data into their respective subspace and to correct the possible noise simultaneously. Recently, it is discovered that the task can be characterized, both theoretically and numerically, by solving a matrix nuclear-norm and a ℓ-mixed norm involved convex minimization problems. The minimization model actually has separable structure in both the objective function and constraint; it thus falls into the framework of the augmented Lagrangian alternating direction approach. In this paper, we propose and investigate an augmented Lagrangian algorithm. We split the augmented Lagrangian function and minimize the subproblems alternatively with one variable by fixing the other one. Moreover, we linearize the subproblem and add a proximal point term to easily derive the closed-form solutions. Global convergence of the proposed algorithm is established under some technical conditions. Extensive experiments on the simulated and the real data verify that the proposed method is very effective and faster than the sate-of-the-art algorithm LRR. [ABSTRACT FROM AUTHOR]

In this paper, we propose, analyze and test primal and dual versions of the alternating direction algorithm for the sparse signal reconstruction from its major noise contained observation data. The algorithm minimizes a convex non-smooth function consisting of the sum of ℓ-norm regularization term and ℓ-norm data fidelity term. We minimize the corresponding augmented Lagrangian function alternatively from either primal or dual forms. Both of the resulting subproblems admit explicit solutions either by using a one-dimensional shrinkage or by an efficient Euclidean projection. The algorithm is easily implementable and it requires only two matrix-vector multiplications per-iteration. The global convergence of the proposed algorithm is established under some technical conditions. The extensions to the non-negative signal recovery problem and the weighted regularization minimization problem are also discussed and tested. Numerical results illustrate that the proposed algorithm performs better than the state-of-the-art algorithm YALL1. [ABSTRACT FROM AUTHOR]

Descent property is very important for an iterative method to be globally convergent. In this paper, we propose a way to construct sufficient descent directions for unconstrained optimization. We then apply the technique to derive a PSB (Powell-Symmetric-Broyden) based method. The PSB based method locally reduces to the standard PSB method with unit steplength. Under appropriate conditions, we show that the PSB based method with Armijo line search or Wolfe line search is globally and superlinearly convergent for uniformly convex problems. We also do some numerical experiments. The results show that the PSB based method is competitive with the standard BFGS method. [ABSTRACT FROM AUTHOR]