Search

In this paper we combine the stochastic variance reduced gradient (SVRG) method [17] with the primal dual fixed point method (PDFP) proposed in [7] to solve a sum of two convex functions and one of which is linearly composite. This type of problems are typically arisen in sparse signal and image reconstruction. The proposed SVRG-PDFP can be seen as a generalization of Prox-SVRG [37] originally designed for the minimization of a sum of two convex functions. Based on some standard assumptions, we propose two variants, one is for strongly convex objective function and the other is for general convex cases. Convergence analysis shows that the convergence rate of SVRG-PDFP is O(1/k) (here k is the iteration number) for general convex objective function and linear for k strongly convex case. Numerical examples on machine learning and CT image reconstruction are provided to show the effectiveness of the algorithms.

In this paper, we propose a second-order continuous primal-dual dynamical system with time-dependent positive damping terms for a separable convex optimization problem with linear equality constraints. By the Lyapunov function approach, we investigate asymptotic properties of the proposed dynamical system as the time $t\to+\infty$. The convergence rates are derived for different choices of the damping coefficients. We also show that the obtained results are robust under external perturbations.

In nonlinearly constrained optimization, penalty methods provide an effective strategy for handling equality constraints, while barrier methods provide an effective approach for the treatment of in...

This paper presents a primal-dual weak Galerkin (PD-WG) finite element method for a class of second order elliptic equations of Fokker-Planck type. The method is based on a variational form where all the derivatives are applied to the test functions so that no regularity is necessary for the exact solution of the model equation. The numerical scheme is designed by using locally constructed weak second order partial derivatives and the weak gradient commonly used in the weak Galerkin context. Optimal order of convergence is derived for the resulting numerical solutions. Numerical results are reported to demonstrate the performance of the numerical scheme., Comment: 26 pages, 10 tables, 3 figures

In this paper, we propose two novel non-stationary first-order primal-dual algorithms to solve nonsmooth composite convex optimization problems. Unlike existing primal-dual schemes where the parameters are often fixed, our methods use pre-defined and dynamic sequences for parameters. We prove that our first algorithm can achieve $\mathcal{O}(1/k)$ convergence rate on the primal-dual gap, and primal and dual objective residuals, where $k$ is the iteration counter. Our rate is on the non-ergodic (i.e., the last iterate) sequence of the primal problem and on the ergodic (i.e., the averaging) sequence of the dual problem, which we call semi-ergodic rate. By modifying the step-size update rule, this rate can be boosted even faster on the primal objective residual. When the problem is strongly convex, we develop a second primal-dual algorithm that exhibits $\mathcal{O}(1/k^2)$ convergence rate on the same three types of guarantees. Again by modifying the step-size update rule, this rate becomes faster on the primal objective residual. Our primal-dual algorithms are the first ones to achieve such fast convergence rate guarantees under mild assumptions compared to existing works, to the best of our knowledge. As byproducts, we apply our algorithms to solve constrained convex optimization problems and prove the same convergence rates on both the objective residuals and the feasibility violation. We still obtain at least $\mathcal{O}(1/k^2)$ rates even when the problem is "semi-strongly" convex. We verify our theoretical results via two well-known numerical examples., Comment: 30 pages, 2 figures

consider primal-dual mixed finite element methods for the solution of the elliptic Cauchy problem, or other related data assimilation problems. The method has a local conservation property. We deri ...

We consider the following single-machine scheduling problem, which is often denoted $1||\sum f_{j}$: we are given $n$ jobs to be scheduled on a single machine, where each job $j$ has an integral processing time $p_j$, and there is a nondecreasing, nonnegative cost function $f_j(C_{j})$ that specifies the cost of finishing $j$ at time $C_{j}$; the objective is to minimize $\sum_{j=1}^n f_j(C_j)$. Bansal \& Pruhs recently gave the first constant approximation algorithm with a performance guarantee of 16. We improve on this result by giving a primal-dual pseudo-polynomial-time algorithm based on the recently introduced knapsack-cover inequalities. The algorithm finds a schedule of cost at most four times the constructed dual solution. Although we show that this bound is tight for our algorithm, we leave open the question of whether the integrality gap of the LP is less than 4. Finally, we show how the technique can be adapted to yield, for any $\epsilon >0$, a $(4+\epsilon )$-approximation algorithm for this problem., Comment: 26 pages. A preliminary version appeared in APPROX 2011. arXiv admin note: text overlap with arXiv:1403.0298

We propose primal-dual active-set (PDAS) methods for solving large-scale instances of an important class of convex quadratic optimization problems (QPs). The iterates of the algorithms are partitions of the index set of variables, where corresponding to each partition there exist unique primal-dual variables that can be obtained by solving a (reduced) linear system. Algorithms of this type have recently received attention when solving certain QPs and linear complementarity problems since, with rapid changes in the active set estimate, they often converge in few iterations. Indeed, as discussed in this paper, convergence in a finite number of iterations is guaranteed when a basic PDAS method is employed to solve certain QPs for which a reduced Hessian of the objective function is (a perturbation of) an $M$-matrix. We propose three PDAS algorithms. The novelty of the algorithms is that they allow inexactness in the (reduced) linear system solves at all partitions except optimal ones. Such a feature is parti...

An interior point method for the structural topology optimization is proposed. The linear systems arising in the method are solved by the conjugate gradient method preconditioned by geometric multigrid. The resulting method is then compared with the so-called optimality condition method, an established technique in topology optimization. This method is also equipped with the multigrid preconditioned conjugate gradient algorithm. We conclude that, for large scale problems, the interior point method with an inexact iterative linear solver is superior to any other variant studied in the paper.

We consider the primal problem of finding the zeros of the sum of a maximal monotone operator and the composition of another maximal monotone operator with a linear continuous operator. By formulat...

International audience; In this article we study a continuous Primal-Dual method proposed by Appleton and Talbot and generalize it to other problems in image processing. We interpret it as an Arrow-Hurwicz method which leads to a better description of the system of PDEs obtained. We show existence and uniqueness of solutions and get a convergence result for the denoising problem. Our analysis also yields new a posteriori estimates.

Three primal-dual interior-point algorithms for homogeneous cone programming are presented. They are a short-step algorithm, a large-update algorithm, and a predictor-corrector algorithm. These algorithms are described and analyzed based on a characterization of homogeneous cones via T-algebras. The analysis shows that the algorithms have polynomial iteration complexity Published version

We study the second-order feasibility cone $\mathcal{F}=\{y\in\mathbb{R}^n:\|My\|\leq g^Ty\}$ for given data $(M,g)$. We construct a new representation for this cone and its dual based on the spectral decomposition of the matrix $M^TM-gg^T$. This representation is used to efficiently solve the problem of projecting an arbitrary point $x\in\mathbb{R}^n$ onto $\mathcal{F}$: $\min_y\{\|y-x\|:\|My\|\leq g^Ty\}$, which aside from theoretical interest also arises as a necessary subroutine in the rescaled perceptron algorithm. We develop a method for solving the projection problem to an accuracy $\varepsilon$, whose computational complexity is bounded by $O(mn^2+n\ln\ln(1/\varepsilon)+n\ln\ln(1/\min\{\operatorname{width}(\mathcal{F}),\operatorname{width}(\mathcal{F}^*)\}))$ operations. Here $\operatorname{width}(\mathcal{F})$ and $\operatorname{width}(\mathcal{F}^*)$ denote the width of $\mathcal{F}$ and $\mathcal{F}^*$, respectively. We also perform computational tests that indicate that the method is extremely efficient in practice.

A first order affine scaling method and two $m$th order affine scaling methods for solving monotone linear complementarity problems (LCPs) are presented. All three methods produce iterates in a wide neighborhood of the central path. The first order method has $O(nL^2(\lognL^2) (\log\lognL^2))$ iteration complexity. If the LCP admits a strict complementary solution, then both the duality gap and the iteration sequence converge superlinearly with Q-order two. If $m=\Omega(\log(\sqrt{n}L))$, then both higher order methods have $O(\sqrt{n})L$ iteration complexity. The Q-order of convergence of one of the methods is $(m+1)$ for problems that admit a strict complementarity solution, while the Q-order of convergence of the other method is $(m+1)/2$ for general monotone LCPs.

We present convergence conditions for a generic primal-dual interior-point algorithm with multiple corrector directions. The corrector directions can be generated by any approach. The search direction is obtained by combining predictor and corrector directions through a small linear program. We also propose a new approach to generate corrector directions. This approach generates directions using information from an appropriately defined Krylov subspace. We propose efficient implementation strategies for our approach that follow the analysis of this paper. Numerical experiments illustrating the features of the proposed approach and its practical usefulness are reported.

We discuss two approximation paradigms that were used to construct many approximation algorithms during the last two decades, the primal-dual schema and the local ratio technique. Recently, primal-dual algorithms were devised by first constructing a local ratio algorithm and then transforming it into a primal-dual algorithm. This was done in the case of the $2$-approximation algorithms for the feedback vertex set problem and in the case of the first primal-dual algorithms for maximization problems. Subsequently, the nature of the connection between the two paradigms was posed as an open question by Williamson [Math. Program., 91 (2002), pp. 447--478]. In this paper we answer this question by showing that the two paradigms are equivalent.

For a conic optimization problem: minimize c*x subject to Ax=b, x in C, we present a geometric relationship between the maximum norms of the level sets of the primal and the inscribed sizes of the level sets of the dual (or the other way around).

An algorithm for efficient solution of optimal control problems with pointwise constraints is introduced. It is based on an active set strategy using primal as well as dual variables. Under certain assumptions a decrease of an appropriately chosen merit function and convergence of the algorithm are proved. Numerical examples for a parabolic optimal control problem demonstrate the efficiency of the proposed method for control-constrained problems.

This paper establishes theorems about the simultaneous variation of right-hand sides and cost coefficients in a linear program from a strictly complementary solution. Some results are extensions of those that have been proven for varying the right-hand side of the primal or the dual, but not both; other results are new. In addition, changes in the optimal partition and what that means in economic terms are related to the basis-driven approach, notably to the theory of compatibility. In addition to new theorems about this relation, the transition graph is extended to provide another visualization of the underlying economics.

This work concerns primal--dual interior-point methods for semidefinite programming (SDP) that use a search direction originally proposed by Helmberg et al. [SIAM J. Optim., 6 (1996), pp. 342--361] and Kojima, Shindoh, and Hara [SIAM J. Optim., 7 (1997), pp. 86--125.] and recently rediscovered by Monteiro [SIAM J. Optim., 7 (1997), pp. 663--678] in a more explicit form. In analyzing these methods, a number of basic equalities and inequalities were developed in [Kojima, Shindoh, and Hara] and also in [Monteiro] through different means and in different forms. In this paper, we give a concise derivation of the key equalities and inequalities for complexity analysis along the exact line used in linear programming (LP), producing basic relationships that have compact forms almost identical to their counterparts in LP. We also introduce a new formulation of the central path and variable-metric measures of centrality. These results provide convenient tools for deriving polynomiality results for primal--dual algorithms extended from LP to SDP using the aforementioned and related search directions. We present examples of such extensions, including the long-step infeasible-interior-point algorithm of Zhang [SIAM J. Optim., 4 (1994), pp. 208--227].

We build on the classical greedy sequential set cover algorithm, in the spirit of the primal-dual schema, to obtain simple parallel approximation algorithms for the set cover problem and its generalizations. Our algorithms use randomization, and our randomized voting lemmas may be of independent interest. Fast parallel approximation algorithms were known before for set cover, though not for the generalizations considered in this paper.

The aim of this paper is two-fold. First, new variants are proposed for the primal-dual steepest descent algorithm as one in the family of primal-dual projected gradient algorithms developed by Zhu and Rockafellar [SIAM J. Optim., 3 (1993), pp. 751–783] for large-scale extended linear-quadratic programming. The variants include a second update scheme for the iterates, where the primal-dual feedback is arranged in a new pattern, as well as alternatives for the “perfect line search” in the original version of the reference. Second, new linear convergence results are proved for all these variants of the algorithm, including the original version as a special case, without the additional assumptions used by Zhu and Rockafellar. For the variants with the second update scheme, a much sharper estimation for the rate of convergence is obtained due to the new primal-dual feedback pattern.

In this paper we consider a class of combined primal-dual and penalty methods often called methods of multipliers. The analysis focuses mainly on the rate of convergence of these methods. It is sho...

In this paper we propose and analyze a class of combined primal–dual and penalty methods for constrained minimization which generalizes the method of multipliers. We provide a convergence and rate of convergence analysis for these methods for the case of a convex programming problem. We prove global convergence in the presence of both exact and inexact unconstrained minimization, and we show that the rate of convergence may be linear or superlinear with arbitrary Q-order of convergence depending on the problem at hand and the form of the penalty function employed.