Search

In this paper, we propose a new algorithm for solving a bilevel equilibrium problem in a real Hilbert space. In contrast to most other projection-type algorithms, which require to solve subproblems at each iteration, the subgradient method proposed in this paper requires only to calculate, at each iteration, two subgradients of convex functions and one projection onto a convex set. Hence, our algorithm has a low computational cost. We prove a strong convergence theorem for the proposed algorithm and apply it for solving the equilibrium problem over the fixed point set of a nonexpansive mapping. Some numerical experiments and comparisons are given to illustrate our results. Also, an application to Nash-Cournot equilibrium models of a semioligopolistic market is presented. [ABSTRACT FROM AUTHOR]

This paper considers the problem of unconstrained minimization of smooth convex functions having Lipschitz continuous gradients with known Lipschitz constant. We recently proposed the optimized gradient method for this problem and showed that it has a worst-case convergence bound for the cost function decrease that is twice as small as that of Nesterov's fast gradient method, yet has a similarly efficient practical implementation. Drori showed recently that the optimized gradient method has optimal complexity for the cost function decrease over the general class of first-order methods. This optimality makes it important to study fully the convergence properties of the optimized gradient method. The previous worst-case convergence bound for the optimized gradient method was derived for only the last iterate of a secondary sequence. This paper provides an analytic convergence bound for the primary sequence generated by the optimized gradient method. We then discuss additional convergence properties of the optimized gradient method, including the interesting fact that the optimized gradient method has two types of worst-case functions: a piecewise affine-quadratic function and a quadratic function. These results help complete the theory of an optimal first-order method for smooth convex minimization. [ABSTRACT FROM AUTHOR]

This paper considers a networked system with a finite number of users and supposes that each user tries to minimize its own private objective function over its own private constraint set. It is assumed that each user's constraint set can be expressed as a fixed point set of a certain quasi-nonexpansive mapping. This enables us to consider the case in which the projection onto the constraint set cannot be computed efficiently. This paper proposes two methods for solving the problem of minimizing the sum of their nondifferentiable, convex objective functions over the intersection of their fixed point sets of quasi-nonexpansive mappings in a real Hilbert space. One method is a parallel subgradient method that can be implemented under the assumption that each user can communicate with other users. The other is an incremental subgradient method that can be implemented under the assumption that each user can communicate with its neighbors. Investigation of the two methods' convergence properties for a constant step size reveals that, with a small constant step size, they approximate a solution to the problem. Consideration of the case in which the step-size sequence is diminishing demonstrates that the sequence generated by each of the two methods strongly converges to the solution to the problem under certain assumptions. Convergence rate analysis of the two methods under certain situations is provided to illustrate the two methods' efficiency. This paper also discusses nonsmooth convex optimization over sublevel sets of convex functions and provides numerical comparisons that demonstrate the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

This paper is concerned with the minimization of an objective that is the sum of a convex function f and an $$\ell _1$$ regularization term. Our interest is in active-set methods that incorporate second-order information about the function f to accelerate convergence. We describe a semismooth Newton framework that can be used to generate a variety of second-order methods, including block active set methods, orthant-based methods and a second-order iterative soft-thresholding method. The paper proposes a new active set method that performs multiple changes in the active manifold estimate at every iteration, and employs a mechanism for correcting these estimates, when needed. This corrective mechanism is also evaluated in an orthant-based method. Numerical tests comparing the performance of three active set methods are presented. [ABSTRACT FROM AUTHOR]

We propose dynamic sampled stochastic approximation (SA) methods for stochastic optimization with a heavy-tailed distribution (with finite 2nd moment). The objective is the sum of a smooth convex function with a convex regularizer. Typically, it is assumed an oracle with an upper bound σ 2 on its variance (OUBV). Differently, we assume an oracle with multiplicative noise. This rarely addressed setup is more aggressive but realistic, where the variance may not be uniformly bounded. Our methods achieve optimal iteration complexity and (near) optimal oracle complexity. For the smooth convex class, we use an accelerated SA method a la FISTA which achieves, given tolerance ε > 0 , the optimal iteration complexity of O (ε - 1 2 ) with a near-optimal oracle complexity of O (ε - 2) [ ln (ε - 1 2 ) ] 2 . This improves upon Ghadimi and Lan (Math Program 156:59–99, 2016) where it is assumed an OUBV. For the strongly convex class, our method achieves optimal iteration complexity of O (ln (ε - 1)) and optimal oracle complexity of O (ε - 1) . This improves upon Byrd et al. (Math Program 134:127–155, 2012) where it is assumed an OUBV. In terms of variance, our bounds are local: they depend on variances σ (x ∗) 2 at solutions x ∗ and the per unit distance multiplicative variance σ L 2 . For the smooth convex class, there exist policies such that our bounds resemble, up to absolute constants, those obtained in the mentioned papers if it was assumed an OUBV with σ 2 : = σ (x ∗) 2 . For the strongly convex class such property is obtained exactly if the condition number is estimated or in the limit for better conditioned problems or for larger initial batch sizes. In any case, if it is assumed an OUBV, our bounds are thus sharper since typically max { σ (x ∗) 2 , σ L 2 } ≪ σ 2 . [ABSTRACT FROM AUTHOR]

The Barzilai-Borwein conjugate gradient methods, which were first proposed by Dai and Kou (Sci China Math 59(8):1511-1524, 2016), are very interesting and very efficient for strictly convex quadratic minimization. In this paper, we present an efficient Barzilai-Borwein conjugate gradient method for unconstrained optimization. Motivated by the Barzilai-Borwein method and the linear conjugate gradient method, we derive a new search direction satisfying the sufficient descent condition based on a quadratic model in a two-dimensional subspace, and design a new strategy for the choice of initial stepsize. A generalized Wolfe line search is also proposed, which is nonmonotone and can avoid a numerical drawback of the original Wolfe line search. Under mild conditions, we establish the global convergence and the R-linear convergence of the proposed method. In particular, we also analyze the convergence for convex functions. Numerical results show that, for the CUTEr library and the test problem collection given by Andrei, the proposed method is superior to two famous conjugate gradient methods, which were proposed by Dai and Kou (SIAM J Optim 23(1):296-320, 2013) and Hager and Zhang (SIAM J Optim 16(1):170-192, 2005), respectively. [ABSTRACT FROM AUTHOR]

In this paper, results on the strong convergence of subgradients of convex functions along a given direction are presented; that is, the relative compactness (with respect to the norm) of the union of subdifferentials of a convex function along a given direction is investigated. [ABSTRACT FROM AUTHOR]

This paper is devoted to the design of an efficient and convergent semi-proximal alternating direction method of multipliers (ADMM) for finding a solution of low to medium accuracy to convex quadratic conic programming and related problems. For this class of problems, the convergent two block semi-proximal ADMM can be employed to solve their primal form in a straightforward way. However, it is known that it is more efficient to apply the directly extended multi-block semi-proximal ADMM, though its convergence is not guaranteed, to the dual form of these problems. Naturally, one may ask the following question: can one construct a convergent multi-block semi-proximal ADMM that is more efficient than the directly extended semi-proximal ADMM? Indeed, for linear conic programming with 4-block constraints this has been shown to be achievable in a recent paper by Sun et al. (, ). Inspired by the aforementioned work and with the convex quadratic conic programming in mind, we propose a Schur complement based convergent semi-proximal ADMM for solving convex programming problems, with a coupling linear equality constraint, whose objective function is the sum of two proper closed convex functions plus an arbitrary number of convex quadratic or linear functions. Our convergent semi-proximal ADMM is particularly suitable for solving convex quadratic semidefinite programming (QSDP) with constraints consisting of linear equalities, a positive semidefinite cone and a simple convex polyhedral set. The efficiency of our proposed algorithm is demonstrated by numerical experiments on various examples including QSDP. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a new iterative algorithm for solving the split equality generalized mixed equilibrium problems. The weak and strong convergence theorems are proved for demi-contractive mappings in real Hilbert spaces. Several special cases are also discussed. As applications, we employ our results to get the convergence results for the split equality convex differentiable optimization problem, the split equality convex minimization problem, and the split equality mixed equilibrium problem. The results in this paper generalize, extend, and unify some recent results in the literature. [ABSTRACT FROM AUTHOR]

In this paper, two iteration processes are used to find the solutions of the mathematical programming for the sum of two convex functions. In infinite Hilbert space, we establish two strong convergence theorems as regards this problem. As applications of our results, we give strong convergence theorems as regards the split feasibility problem with modified CQ method, strong convergence theorem as regards the lasso problem, and strong convergence theorems for the mathematical programming with a modified proximal point algorithm and a modified gradient-projection method in the infinite dimensional Hilbert space. We also apply our result on the lasso problem to the image deblurring problem. Some numerical examples are given to demonstrate our results. The main result of this paper entails a unified study of many types of optimization problems. Our algorithms to solve these problems are different from any results in the literature. Some results of this paper are original and some results of this paper improve, extend, and unify comparable results in existence in the literature. [ABSTRACT FROM AUTHOR]

In this paper, we focus on restoring blurred images with additive noise and multiplicative noise. Based on the statistical property of the Gamma noise, we use a quadratic penalty function technique in order to obtain a component-wise convex model. We employ the alternating minimization method to solve the proposed model and study the convergence of this method. Numerical experiments demonstrate that the proposed model has superior performance in terms of the relative error and the peak signal-to-noise ratio than a state-of-the-art model for restoring blurred images with additive noise and multiplicative noise. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new modified proximal point algorithm for finding a common element of the set of common minimizers of a finite family of convex and lower semi-continuous functions and the set of common fixed points of a finite family of nonexpansive mappings in complete CAT(0) spaces, and prove some convergence theorems of the proposed algorithm under suitable conditions. A numerical example is presented to illustrate the proposed method and convergence result. Our results improve and extend the corresponding results existing in the literature. [ABSTRACT FROM AUTHOR]

In this paper, a modified BFGS algorithm is proposed for unconstrained optimization. The proposed algorithm has the following properties: (i) a nonmonotone line search technique is used to obtain the step size $\alpha_{k}$ to improve the effectiveness of the algorithm; (ii) the algorithm possesses not only global convergence but also superlinear convergence for generally convex functions; (iii) the algorithm produces better numerical results than those of the normal BFGS method. [ABSTRACT FROM AUTHOR]

The main result of this paper is to prove the strong convergence of the sequence generated by the proximal point algorithm of Halpern type to a zero of a maximal monotone operator under the suitable assumptions on the parameters and error. The results extend some of the previous results or give some different conditions for convergence of the sequence. It is also indicated that when the maximal monotone operator is the subdifferential of a convex, proper, and lower semicontinuous function, the results extend all previous results in the literature. We also prove the boundedness of the sequence generated by the algorithm with a weak coercivity condition defined in the paper and without any additional assumptions on the parameters. [ABSTRACT FROM AUTHOR]

The Douglas-Rachford algorithm is a very popular splitting technique for finding a zero of the sum of two maximally monotone operators. The behaviour of the algorithm remains mysterious in the general inconsistent case, i.e., when the sum problem has no zeros. However, more than a decade ago, it was shown that in the (possibly inconsistent) convex feasibility setting, the shadow sequence remains bounded and its weak cluster points solve a best approximation problem. In this paper, we advance the understanding of the inconsistent case significantly by providing a complete proof of the full weak convergence in the convex feasibility setting. In fact, a more general sufficient condition for the weak convergence in the general case is presented. Our proof relies on a new convergence principle for Fejér monotone sequences. Numerous examples illustrate our results. [ABSTRACT FROM AUTHOR]

In this paper, we generalize the proximal point algorithm to complete CAT(0) spaces and show that the sequence generated by the proximal point algorithm w-converges to a zero of the maximal mono-tone operator. Also, we prove that if f : X →] - ∞, +∞] is a proper, convex and lower semicontinuous function on the complete CAT(0) space f, then the proximal point algorithm w-converges to a zero of the sub-differential of f, i.e., a minimizer of f. Some strong convergence results (convergence in metric) are also presented with additional assumptions on the monotone operator and the convex function f. [ABSTRACT FROM AUTHOR]

In this paper, we prove new complexity bounds for methods of convex optimization based only on computation of the function value. The search directions of our schemes are normally distributed random Gaussian vectors. It appears that such methods usually need at most n times more iterations than the standard gradient methods, where n is the dimension of the space of variables. This conclusion is true for both nonsmooth and smooth problems. For the latter class, we present also an accelerated scheme with the expected rate of convergence $$O\Big ({n^2 \over k^2}\Big )$$ , where k is the iteration counter. For stochastic optimization, we propose a zero-order scheme and justify its expected rate of convergence $$O\Big ({n \over k^{1/2}}\Big )$$ . We give also some bounds for the rate of convergence of the random gradient-free methods to stationary points of nonconvex functions, for both smooth and nonsmooth cases. Our theoretical results are supported by preliminary computational experiments. [ABSTRACT FROM AUTHOR]

The proximal alternating direction method of multipliers (P-ADMM) is an efficient first-order method for solving the separable convex minimization problems. Recently, He et al. have further studied the P-ADMM and relaxed the proximal regularization matrix of its second subproblem to be indefinite. This is especially significant in practical applications since the indefinite proximal matrix can result in a larger step size for the corresponding subproblem and thus can often accelerate the overall convergence speed of the P-ADMM. In this paper, without the assumptions that the feasible set of the studied problem is bounded or the objective function's component $\theta_{i}(\cdot)$ of the studied problem is strongly convex, we prove the worst-case $\mathcal{O}(1/t)$ convergence rate in an ergodic sense of the P-ADMM with a general Glowinski relaxation factor $\gamma\in(0,\frac{1+\sqrt{5}}{2})$ , which is a supplement of the previously known results in this area. Furthermore, some numerical results on compressive sensing are reported to illustrate the effectiveness of the P-ADMM with indefinite proximal regularization. [ABSTRACT FROM AUTHOR]

In this paper, we show that when the alternating direction method of multipliers (ADMM) is extended directly to the 3-block separable convex minimization problems, it is convergent if one block in the objective possesses sub-strong monotonicity which is weaker than strong convexity. In particular, we estimate the globally linear convergence rate of the direct extension of ADMM measured by the iteration complexity under some additional conditions. [ABSTRACT FROM AUTHOR]

We introduce new optimized first-order methods for smooth unconstrained convex minimization. Drori and Teboulle (Math Program 145(1-2):451-482, 2014. doi:) recently described a numerical method for computing the N-iteration optimal step coefficients in a class of first-order algorithms that includes gradient methods, heavy-ball methods (Polyak in USSR Comput Math Math Phys 4(5):1-17, 1964. doi:), and Nesterov's fast gradient methods (Nesterov in Sov Math Dokl 27(2):372-376, 1983; Math Program 103(1):127-152, 2005. doi:). However, the numerical method in Drori and Teboulle (2014) is computationally expensive for large N, and the corresponding numerically optimized first-order algorithm in Drori and Teboulle (2014) requires impractical memory and computation for large-scale optimization problems. In this paper, we propose optimized first-order algorithms that achieve a convergence bound that is two times smaller than for Nesterov's fast gradient methods; our bound is found analytically and refines the numerical bound in Drori and Teboulle (2014). Furthermore, the proposed optimized first-order methods have efficient forms that are remarkably similar to Nesterov's fast gradient methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new fast normal-based interpolating subdivision scheme for curve and surface design. Different from the 4-points interpolating subdivision scheme, it is based on cubic Bezier curves and the normal vectors are used to generate a circle. Both a convex edge and an inflexion edge can be subdivided into convex sub-edges and then generate smooth curves. Under proper angle conditions, this subdivision scheme converges and the limit curve will be $$\hbox {G}^{1}$$ smoothness. When applying it to subdivide surface on triangle/quadrilateral meshes, we use the normal vectors and have no need to consider the meshes neighboring to the current surface elements. Such advantage leads to that the subdivision scheme has fast rendering speed without changing the topology of the meshes. Subdivision examples and results by our scheme are illustrated and meantime is compared with those generated by other well-known schemes. It shows that this scheme can generate a more smooth curve based on both a convex edge and an inflexion edge, and the limit surface has better smoothness than those of other interpolating schemes. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to use the modified block iterative method to propose an algorithm for solving the convex feasibility problems for an infinite family of quasi-f-asymptotically nonexpansive mappings. Under suitable conditions some strong convergence theorems are established in uniformly smooth and strictly convex Banach spaces with Kadec-Klee property. The results presented in the paper improve and extend some recent results. [ABSTRACT FROM AUTHOR]

The alternating direction method of multipliers (ADMM) is now widely used in many fields, and its convergence was proved when two blocks of variables are alternatively updated. It is strongly desirable and practically valuable to extend the ADMM directly to the case of a multi-block convex minimization problem where its objective function is the sum of more than two separable convex functions. However, the convergence of this extension has been missing for a long time-neither an affirmative convergence proof nor an example showing its divergence is known in the literature. In this paper we give a negative answer to this long-standing open question: The direct extension of ADMM is not necessarily convergent. We present a sufficient condition to ensure the convergence of the direct extension of ADMM, and give an example to show its divergence. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new nonmonotone adaptive retrospective Trust Region (TR) method for solving unconstrained optimization problems. Inspired by the retrospective ratio proposed in Bastin et al. (Math Program Ser A 123:395-418, ), a new nonmonotone TR ratio is introduced based on a convex combination of the nonmonotone classical and retrospective ratios. Due to the value of the new ratio, the TR radius is updated adaptively by a variant of the rule as proposed in Shi and Guo (J Comput Appl Math 213:509-520, ). Global convergence property of the new algorithm, as well as its superlinear convergence rate, is established under some standard assumptions. Numerical results on some test problems show the efficiency and effectiveness of the new method in practice, too. [ABSTRACT FROM AUTHOR]

In this paper, a new nonlinear conjugate gradient method is proposed, whose search direction can be viewed as a simple approximation to that of the memoryless BFGS method. The search direction of the proposed method satisfies the sufficient descent property regardless of line search. Global convergence properties of the new method are explored on uniformly convex functions and general functions with the standard Wolfe line search. Numerical experiments are done to test the efficiency of the new method, which implies the new method is promising. [ABSTRACT FROM AUTHOR]

In this paper, we present new methods for black-box convex minimization. They do not need to know in advance the actual level of smoothness of the objective function. Their only essential input parameter is the required accuracy of the solution. At the same time, for each particular problem class they automatically ensure the best possible rate of convergence. We confirm our theoretical results by encouraging numerical experiments, which demonstrate that the fast rate of convergence, typical for the smooth optimization problems, sometimes can be achieved even on nonsmooth problem instances. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive sizing BFGS method for unconstrained optimization is proposed, whose scaling factor is automatically chosen by the signature of some terms of the approximate model's curvature. The scaling factor is always chosen less than or equal to one as required by the convergence property of the self-scaling BFGS method of Al-Baali (Comput Optim Appl 9:191-203, ), while the choosing strategy can ensure the sufficient positive definiteness of the updating matrices. Under mild conditions, the global convergence properties of Al-Baali on convex functions are proved. Numerical results on some test problems show the proposed method is competitive with its counterparts provided that the scaling factor is not too small. [ABSTRACT FROM AUTHOR]

We consider a general class of convex optimization problems in which one seeks to minimize a strongly convex function over a closed and convex set which is by itself an optimal set of another convex problem. We introduce a gradient-based method, called the minimal norm gradient method, for solving this class of problems, and establish the convergence of the sequence generated by the algorithm as well as a rate of convergence of the sequence of function values. The paper ends with several illustrating numerical examples. [ABSTRACT FROM AUTHOR]

Row-action methods play an important role in tomographic image reconstruction. Many such methods can be viewed as incremental gradient methods for minimizing a sum of a large number of convex functions, and despite their relatively poor global rate of convergence, these methods often exhibit fast initial convergence which is desirable in applications where a low-accuracy solution is acceptable. In this paper, we propose relaxed variants of a class of incremental proximal gradient methods, and these variants generalize many existing row-action methods for tomographic imaging. Moreover, they allow us to derive new incremental algorithms for tomographic imaging that incorporate different types of prior information via regularization. We demonstrate the efficacy of the approach with some numerical examples. [ABSTRACT FROM AUTHOR]

This paper is aimed to extend a certain damped technique, suitable for the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, to the limited memory BFGS method in the case of the large-scale unconstrained optimization. It is shown that the proposed technique maintains the global convergence property on uniformly convex functions for the limited memory BFGS method. Some numerical results are described to illustrate the important role of the damped technique. Since this technique enforces safely the positive definiteness property of the BFGS update for any value of the steplength, we also consider only the first Wolfe-Powell condition on the steplength. Then, as for the backtracking framework, only one gradient evaluation is performed on each iteration. It is reported that the proposed damped methods work much better than the limited memory BFGS method in several cases. [ABSTRACT FROM AUTHOR]

Conditional Value at Risk (CVaR) has been recently used to approximate a chance constraint. In this paper, we study the convergence of stationary points, when sample average approximation (SAA) method is applied to a CVaR approximated joint chance constrained stochastic minimization problem. Specifically, we prove under some moderate conditions that optimal solutions and stationary points, obtained from solving sample average approximated problems, converge with probability one to their true counterparts. Moreover, by exploiting the recent results on large deviation of random functions and sensitivity results for generalized equations, we derive exponential rate of convergence of stationary points. The discussion is also extended to the case, when CVaR approximation is replaced by a difference of two convex functions (DC-approximation). Some preliminary numerical test results are reported. [ABSTRACT FROM AUTHOR]

A new convex feasibility problem, the split equality problem (SEP), has been proposed by Moudafi and Byrne. The SEP was solved through the ACQA and ARCQA algorithms. In this paper the SEPs are extended to infinite-dimensional SEPs in Hilbert spaces and we established the strong convergence of a proposed algorithm to a solution of general split equality problems (GSEPs). [ABSTRACT FROM AUTHOR]

In this paper, we present two Douglas-Rachford inspired iteration schemes which can be applied directly to N-set convex feasibility problems in Hilbert space. Our main results are weak convergence of the methods to a point whose nearest point projections onto each of the N sets coincide. For affine subspaces, convergence is in norm. Initial results from numerical experiments, comparing our methods to the classical (product-space) Douglas-Rachford scheme, are promising. [ABSTRACT FROM AUTHOR]

In this paper we analyze several new methods for solving optimization problems with the objective function formed as a sum of two terms: one is smooth and given by a black-box oracle, and another is a simple general convex function with known structure. Despite the absence of good properties of the sum, such problems, both in convex and nonconvex cases, can be solved with efficiency typical for the first part of the objective. For convex problems of the above structure, we consider primal and dual variants of the gradient method (with convergence rate $$O\left({1 \over k}\right)$$), and an accelerated multistep version with convergence rate $$O\left({1 \over k^2}\right)$$, where $$k$$ is the iteration counter. For nonconvex problems with this structure, we prove convergence to a point from which there is no descent direction. In contrast, we show that for general nonsmooth, nonconvex problems, even resolving the question of whether a descent direction exists from a point is NP-hard. For all methods, we suggest some efficient 'line search' procedures and show that the additional computational work necessary for estimating the unknown problem class parameters can only multiply the complexity of each iteration by a small constant factor. We present also the results of preliminary computational experiments, which confirm the superiority of the accelerated scheme. [ABSTRACT FROM AUTHOR]

Consider the BFGS quasi-Newton method applied to a general non-convex function that has continuous second derivatives. This paper aims to construct a four-dimensional example such that the BFGS method need not converge. The example is perfect in the following sense: (a) All the stepsizes are exactly equal to one; the unit stepsize can also be accepted by various line searches including the Wolfe line search and the Arjimo line search; (b) The objective function is strongly convex along each search direction although it is not in itself. The unit stepsize is the unique minimizer of each line search function. Hence the example also applies to the global line search and the line search that always picks the first local minimizer; (c) The objective function is polynomial and hence is infinitely continuously differentiable. If relaxing the convexity requirement of the line search function; namely, (b) we are able to construct a relatively simple polynomial example. [ABSTRACT FROM AUTHOR]

The aim of this paper is to establish the stability of weak efficient, efficient and Henig proper efficient sets of a vector optimization problem, using quasiconvex and related functions. We establish the Kuratowski-Painlevé set-convergence of the minimal solution sets of a family of perturbed problems to the corresponding minimal solution set of the vector problem, where the perturbations are performed on both the objective function and the feasible set. This convergence is established by using gamma convergence of the sequence of the perturbed objective functions and Kuratowski-Painlevé set-convergence of the sequence of the perturbed feasible sets. The solution sets of the vector problem are characterized in terms of the solution sets of a scalar problem, where the scalarization function satisfies order preserving and order representing properties. This characterization is further used to establish the Kuratowski-Painlevé set-convergence of the solution sets of a family of scalarized problems to the solution sets of the vector problem. [ABSTRACT FROM AUTHOR]

This paper is about the minimization of Lipschitz-continuous and strongly convex functions over integer points in polytopes. Our results are related to the rate of convergence of a black-box algorithm that iteratively solves special quadratic integer problems with a constant approximation factor. Despite the generality of the underlying problem, we prove that we can find efficiently, with respect to our assumptions regarding the encoding of the problem, a feasible solution whose objective function value is close to the optimal value. We also show that this proximity result is the best possible up to a factor polynomial in the encoding length of the problem. [ABSTRACT FROM AUTHOR]

In this paper, we obtain some results on the boundedness and asymptotic behavior of the sequence generated by the proximal point algorithm without summability assumption on the error sequence. We also study the rate of convergence to minimum value of a proper, convex, and lower semicontinuous function. Finally, we consider the proximal point algorithm for solving equilibrium problems. [ABSTRACT FROM AUTHOR]

Penalty methods are very efficient in finding an optimal solution to constrained optimization problems. In this paper, we present an objective penalty function with two penalty parameters for inequality constrained bilevel programming under the convexity assumption to the lower level problem. Under some conditions, an optimal solution to a bilevel programming defined by the objective penalty function is proved to be an optimal solution to the original bilevel programming. Moreover, based on the objective penalty function, an algorithm is developed to obtain an optimal solution to the original bilevel programming, with its convergence proved under some conditions. [ABSTRACT FROM AUTHOR]

In this paper, we propose and analyze an algorithm that couples the gradient method with a general exterior penalization scheme for constrained or hierarchical minimization of convex functions in Hilbert spaces. We prove that a proper but simple choice of the step sizes and penalization parameters guarantees the convergence of the algorithm to solutions for the optimization problem. We also establish robustness and stability results that account for numerical approximation errors, discuss implementation issues and provide examples in finite and infinite dimension. [ABSTRACT FROM AUTHOR]

We consider the problem of the approximation of regular convex bodies in ℝ by level surfaces of convex algebraic polynomials. Hammer (in Mathematika 10, 67-71, ) verified that any convex body in ℝ can be approximated by a level surface of a convex algebraic polynomial. In Jaen J. Approx. 1, 97-109, and subsequently in J. Approx. Theory 162, 628-637, a quantitative version of Hammer's approximation theorem was given by showing that the order of approximation of convex bodies by convex algebraic level surfaces of degree n is $\frac{1}{n}$. Moreover, it was also shown that whenever the convex body is not regular (that is, there exists a point on its boundary at which the convex body possesses two distinct supporting hyperplanes), then $\frac{1}{n}$ is essentially the sharp rate of approximation. This leads to the natural question whether this rate of approximation can be improved further when the convex body is regular. In this paper we shall give an affirmative answer to this question. It turns out that for regular convex bodies a o(1/ n) rate of convergence holds. In addition, if the body satisfies the condition of C- smoothness the rate of approximation is $O(\frac{1}{n^{2}})$. [ABSTRACT FROM AUTHOR]

We propose a family of convergent double-loop algorithms which minimize the TRW free energy. These algorithms are based on the concave convex procedure (CCCP) so we call them TRW-CCCP. Our formulation includes many free parameters which specify an infinite number of decompositions of the TRW free energy into convex and concave parts. TRW-CCCP is guaranteed to converge to the global minima for any settings of these free parameters, including adaptive settings if they satisfy conditions defined in this paper. We show that the values of these free parameters control the speed of convergence of the the inner and outer loops in TRW-CCCP. We performed experiments on a two-dimensional Ising model observing that TRW-CCCP converges to the global minimum of the TRW free energy and that the convergence rate depends on the parameter settings. We compare with the original message passing algorithm (TRW-BP) by varying the difficulty of the problem (by adjusting the energy function) and the number of iterations in the inner loop of TRW-CCCP. We show that on difficult problems TRW-CCCP converges faster than TRW-BP (in terms of total number of iterations) if few inner loop iterations are used. [ABSTRACT FROM AUTHOR]

In this paper, we use the auxiliary principle technique to suggest and analyze an implicit iterative method for solving bifunction variational inequalities. We also study the convergence criteria of this new method under pseudomonotonicity condition. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to introduce and investigate two kinds of iterative algorithms for the problem of finding zeros of maximal monotone operators. Weak and strong convergence theorems are established in a real Hilbert space. As applications, we consider a problem of finding a minimizer of a convex function. [ABSTRACT FROM AUTHOR]

We consider a distributed multi-agent network system where the goal is to minimize a sum of convex objective functions of the agents subject to a common convex constraint set. Each agent maintains an iterate sequence and communicates the iterates to its neighbors. Then, each agent combines weighted averages of the received iterates with its own iterate, and adjusts the iterate by using subgradient information (known with stochastic errors) of its own function and by projecting onto the constraint set. The goal of this paper is to explore the effects of stochastic subgradient errors on the convergence of the algorithm. We first consider the behavior of the algorithm in mean, and then the convergence with probability 1 and in mean square. We consider general stochastic errors that have uniformly bounded second moments and obtain bounds on the limiting performance of the algorithm in mean for diminishing and non-diminishing stepsizes. When the means of the errors diminish, we prove that there is mean consensus between the agents and mean convergence to the optimum function value for diminishing stepsizes. When the mean errors diminish sufficiently fast, we strengthen the results to consensus and convergence of the iterates to an optimal solution with probability 1 and in mean square. [ABSTRACT FROM AUTHOR]

In this paper, we scale the quasiNewton equation and propose a spectral scaling BFGS method. The method has a good selfcorrecting property and can improve the behavior of the BFGS method. Compared with the standard BFGS method, the single-step convergence rate of the spectral scaling BFGS method will not be inferior to that of the steepest descent method when minimizing an n-dimensional quadratic function. In addition, when the method with exact line search is applied to minimize an n-dimensional strictly convex function, it terminates within n steps. Under appropriate conditions, we show that the spectral scaling BFGS method with Wolfe line search is globally and R-linear convergent for uniformly convex optimization problems. The reported numerical results show that the spectral scaling BFGS method outperforms the standard BFGS method. [ABSTRACT FROM AUTHOR]

The regularized Newton method (RNM) is one of the efficient solution methods for the unconstrained convex optimization. It is well-known that the RNM has good convergence properties as compared to the steepest descent method and the pure Newton’s method. For example, Li, Fukushima, Qi and Yamashita showed that the RNM has a quadratic rate of convergence under the local error bound condition. Recently, Polyak showed that the global complexity bound of the RNM, which is the first iteration k such that | ∇ f( x k)|≤ ε, is O( ε−4), where f is the objective function and ε is a given positive constant. In this paper, we consider a RNM extended to the unconstrained “nonconvex” optimization. We show that the extended RNM (E-RNM) has the following properties. (a) The E-RNM has a global convergence property under appropriate conditions. (b) The global complexity bound of the E-RNM is O( ε−2) if ∇2 f is Lipschitz continuous on a certain compact set. (c) The E-RNM has a superlinear rate of convergence under the local error bound condition. [ABSTRACT FROM AUTHOR]

Abstract  In this paper we present adaptive algorithms for solving the uniform continuous piecewise affine approximation problem (UCPA) in the case of Lipschitz or convex functions. The algorithms are based on the tree approximation (adaptive splitting) procedure. The uniform convergence is achieved by means of global optimization techniques for obtaining tight upper bounds of the local error estimate (splitting criterion). We give numerical results in the case of the function distance to 2D and 3D geometric bodies. The resulting trees can retrieve the values of the target function in a fast way. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to introduce an iterative method for finding solutions of a general system of variational inclusions with inverse-strongly accretive mappings. Strong convergence theorems are established in uniformly convex and 2-uniformly smooth Banach spaces. [ABSTRACT FROM AUTHOR]

In this paper a new hybrid conjugate gradient algorithm is proposed and analyzed. The parameter β k is computed as a convex combination of the Polak-Ribière-Polyak and the Dai-Yuan conjugate gradient algorithms, i.e. β=(1− θ k) β+ θ k β. The parameter θ k in the convex combination is computed in such a way that the conjugacy condition is satisfied, independently of the line search. The line search uses the standard Wolfe conditions. The algorithm generates descent directions and when the iterates jam the directions satisfy the sufficient descent condition. Numerical comparisons with conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that this hybrid computational scheme outperforms the known hybrid conjugate gradient algorithms. [ABSTRACT FROM AUTHOR]