Search

In this paper, we present a stochastic augmented Lagrangian approach on (possibly infinite-dimensional) Riemannian manifolds to solve stochastic optimization problems with a finite number of deterministic constraints. We investigate the convergence of the method, which is based on a stochastic approximation approach with random stopping combined with an iterative procedure for updating Lagrange multipliers. The algorithm is applied to a multi-shape optimization problem with geometric constraints and demonstrated numerically. [ABSTRACT FROM AUTHOR]

This paper considers a risk-averse Markov decision process (MDP) with non-risk constraints as a dynamic optimization framework to ensure robustness against unfavorable outcomes in high-stakes sequential decision-making situations such as disaster response. In this regard, strong duality is proved while making no assumptions on the problem's convexity. This is necessary for some real-world issues, e.g., in the case of deprivation costs in the context of disaster relief, where convexity cannot be ensured. Our theoretical results imply that the problem can be exactly solved in a dual domain where it becomes convex. Based on our duality results, an augmented Lagrangian-based constraint handling mechanism is also developed for risk-averse reinforcement learning algorithms. The mechanism is proved to be theoretically convergent. Finally, we have also empirically established the convergence of the mechanism using a multi-stage disaster response relief allocation problem while using a fixed negative reward scheme as a benchmark. [ABSTRACT FROM AUTHOR]

We propose a new alternating direction method of multipliers (ADMM) with an optimal parameter for the unilateral obstacle problem. We first use the five-point difference scheme to discretize the problem. Then, we present an augmented Lagrangian by introducing an auxiliary unknown, and an ADMM is applied to the corresponding saddle-point problem. Through eliminating the primal and auxiliary unknowns, a pure dual algorithm is then used. The convergence of the proposed method is analyzed, and a simple strategy is presented for selecting the optimal parameter, with the largest and smallest eigenvalues of the iterative matrix. Several numerical experiments confirm the theoretical findings of this study. [ABSTRACT FROM AUTHOR]

We address the problem of minimizing a smooth function under smooth equality constraints. Under regularity assumptions on these constraints, we propose a notion of approximate first- and second-order critical point which relies on the geometric formalism of Riemannian optimization. Using a smooth exact penalty function known as Fletcher's augmented Lagrangian, we propose an algorithm to minimize the penalized cost function which reaches ε -approximate second-order critical points of the original optimization problem in at most O (ε - 3) iterations. This improves on current best theoretical bounds. Along the way, we show new properties of Fletcher's augmented Lagrangian, which may be of independent interest. [ABSTRACT FROM AUTHOR]

In this two-part study, we develop a general theory of the so-called exact augmented Lagrangians for constrained optimization problems in Hilbert spaces. In contrast to traditional nonsmooth exact penalty functions, these augmented Lagrangians are continuously differentiable for smooth problems and do not suffer from the Maratos effect, which makes them especially appealing for applications in numerical optimization. Our aim is to present a detailed study of various theoretical properties of exact augmented Lagrangians and discuss several applications of these functions to constrained variational problems, problems with PDE constraints, and optimal control problems. The first paper is devoted to a theoretical analysis of an exact augmented Lagrangian for optimization problems in Hilbert spaces. We obtain several useful estimates of this augmented Lagrangian and its gradient, and present several types of sufficient conditions for KKT-points of a constrained problem corresponding to locally/globally optimal solutions to be local/global minimizers of the exact augmented Lagrangian. [ABSTRACT FROM AUTHOR]

The dislocation hyperbolic augmented Lagrangian algorithm (DHALA) is a new approach to the hyperbolic augmented Lagrangian algorithm (HALA). DHALA is designed to solve convex nonlinear programming problems. We guarantee that the sequence generated by DHALA converges towards a Karush-Kuhn-Tucker point. We are going to observe that DHALA has a slight computational advantage in solving the problems over HALA. Finally, we will computationally illustrate our theoretical results. [ABSTRACT FROM AUTHOR]

We discuss several algorithms for solving a network optimization problem of simultaneous routing and bandwidth allocation in green networks in a decomposed way, based on the augmented Lagrangian. The problem is difficult due to the nonconvexity caused by binary routing variables. The chosen algorithms, which are several versions of the Multiplier Method, including the Alternating Direction Method of Multipliers (ADMM), have been implemented in Python and tested on several networks' data. We derive theoretical formulations for the inequality constraints of the Bertsekas, Tatjewski and SALA methods, formulated originally for problems with equality constraints. We also introduce some modifications to the Bertsekas and Tatjewski methods, without which they do not work for an MINLP problem. The final comparison of the performance of these algorithms shows a significant advantage of the augmented Lagrangian algorithms, using decomposition for big problems. In our particular case of the simultaneous routing and bandwidth allocation problem, these algorithms seem to be the best choice. [ABSTRACT FROM AUTHOR]

The primary goal of this paper is to provide an efficient solution algorithm based on the augmented Lagrangian framework for optimization problems with a stochastic objective function and deterministic constraints. Our main contribution is combining the augmented Lagrangian framework with adaptive sampling, resulting in an efficient optimization methodology validated with practical examples. To achieve the presented efficiency, we consider inexact solutions for the augmented Lagrangian subproblems, and through an adaptive sampling mechanism, we control the variance in the gradient estimates. Furthermore, we analyze the theoretical performance of the proposed scheme by showing equivalence to a gradient descent algorithm on a Moreau envelope function, and we prove sublinear convergence for convex objectives and linear convergence for strongly convex objectives with affine equality constraints. The worst-case sample complexity of the resulting algorithm, for an arbitrary choice of penalty parameter in the augmented Lagrangian function, is O (ϵ − 3 − δ) , where ϵ > 0 is the expected error of the solution and δ > 0 is a user-defined parameter. If the penalty parameter is chosen to be O (ϵ − 1) , we demonstrate that the result can be improved to O (ϵ − 2) , which is competitive with the other methods employed in the literature. Moreover, if the objective function is strongly convex with affine equality constraints, we obtain O (ϵ − 1 log ⁡ (1 / ϵ)) complexity. Finally, we empirically verify the performance of our adaptive sampling augmented Lagrangian framework in machine learning optimization and engineering design problems, including topology optimization of a heat sink with environmental uncertainty. [ABSTRACT FROM AUTHOR]

This article proposes an augmented Lagrangian-based topology optimization approach to minimize the volume fraction subject to multiple nodal displacement constraints. The proposed method puts the multiple constraint equations in the objective function and transforms it into a sequence of unconstrained optimization problems. This study explains the theoretical aspects of the employed augmented Lagrangian approach in depth. Sensitivity expression in terms of design variables is identified by exercising the differentiate-then-discretize mechanism, and utilizes a moving asymptote algorithm to sort out a sequence of optimization subproblems. The optimized results are compared and analysed with those from conventional aggregation-based topology optimization. However, the predefined aggregation approach indicates a dependency on the supplied parameters. The numerical two- and three-dimensional examples reveal the viability and reliability of the suggested augmented Lagrangian scheme, which shows advantages in terms of robustness and efficiency. [ABSTRACT FROM AUTHOR]

By revisiting some popular operator splitting ideas, we present several classes of splitting schemes based on the Lagrangian, primal–dual and hybrid formulations, from which can be recovered many existing algorithms, including alternating direction method of multipliers and primal–dual hybrid gradient algorithms. In particular, we show that the generalized proximal point framework is at the root of many past and recent splitting algorithms allowing for an elementary convergence analysis of these methods through a unified scheme. The numerical tests on constrained total variation minimization show that the proposed algorithms could offer more freedom in parameter selections and, thus, achieve faster convergence speed. [ABSTRACT FROM AUTHOR]

Currently, fatigue-resistance topology optimization has received ever increasing attention, in which most of the literature considers this issue as a simple extension of stress-based topology optimization. However, previous approaches may not be applicable when considering general loads, as the conventional uniaxial rainflow counting method, commonly employed in prior studies, can result in significant errors. Furthermore, the inclusion of general loads introduces additional nonlinearity to fatigue-resistant topology optimization, posing challenges in identifying the optimal solution. To this end, a novel methodology for fatigue-resistance topology optimization considering general loads is proposed in this paper. The independent rainflow counting method is utilized during the process of structural damage estimation. The damage penalization model is subsequently adopted to reduce the nonlinearity by scaling the value of fatigue damage. To illustrate the necessity of an independent rainflow counting method, an example of a double L-shaped structure subjected to general loads is presented. The augmented Lagrangian (AL) approach is introduced to transform numerous damage constraint equations into the objective function, generating a sequence of box-constrained optimization sub-problems. After employing the typical SIMP technique, the relative sensitivities of the AL function regarding design variables are derived, which facilitates the efficient solution using the method of moving asymptotes (MMA). Through 2D and 3D numerical tests, the effectiveness of the proposed method is validated in comparison to the traditional method. Further investigation is conducted into the influences of general loads, damage penalization model, and manufacturing error robustness. In addition, the fatigue-resistance performance of a bearing support of a wind turbine is improved by the suggested approach, and its overall weight is decreased by 25.40%. The proposed method addresses the nonlinear and localized nature of fatigue-resistant topology optimization more efficiently. The results indicate that the proposed method can develop a lightweight design for structures under general loads. [ABSTRACT FROM AUTHOR]

The value at risk (VaR) and the conditional value at risk (CVaR) are two popular risk measures to hedge against the uncertainty of data. In this paper, we provide a computational toolbox for solving high-dimensional sparse linear regression problems under either VaR or CVaR measures, the former being nonconvex and the latter convex. Unlike the empirical risk (neutral) minimization models in which the overall losses are decomposable across data, the aforementioned risk-sensitive models have nonseparable objective functions so that typical first order algorithms are not easy to scale. We address this scaling issue by adopting a semismooth Newton-based proximal augmented Lagrangian method of the convex CVaR linear regression problem. The matrix structures of the Newton systems are carefully explored to reduce the computational cost per iteration. The method is further embedded in a majorization–minimization algorithm as a subroutine to tackle the nonconvex VaR-based regression problem. We also discuss an adaptive sieving strategy to iteratively guess and adjust the effective problem dimension, which is particularly useful when a solution path associated with a sequence of tuning parameters is needed. Extensive numerical experiments on both synthetic and real data demonstrate the effectiveness of our proposed methods. In particular, they are about 53 times faster than the commercial package Gurobi for the CVaR-based sparse linear regression with 4,265,669 features and 16,087 observations. History: Accepted by Antonio Frangioni, Area Editor for Design & Analysis of Algorithms–Continuous. Funding: This work was supported in part by the NSF, the Division of Computing and Communication Foundations [Grant 2153352], the National Natural Science Foundation of China [Grant 12271187], and the Hong Kong Research Grant Council [Grant 15304019]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information (https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1282) as well as from the IJOC GitHub software repository (https://github.com/INFORMSJoC/2022.0012) at (http://dx.doi.org/10.5281/zenodo.7483279). [ABSTRACT FROM AUTHOR]

The effects of an included temperature field in the contact process between a piezoelectric body and a rigid foundation with thermal and electrical conductivity are discussed. The constitutive relation of the material is assumed to be thermo-electro-elastic and involves the nonlinear elastic constitutive Hencky's law. The frictional contact is modeled with Signorini's conditions, the regularized Coulomb law, and the regularized electrical and thermal conductivity conditions. The resulting thermo-electromechanical model includes the temperature field as an additional state variable to take into account thermal effects alongside with those of the piezoelectric. The existence of the unique weak solution to the problem is established by using Schauder's fixed point theorem combined with arguments from the theory of variational inequalities involving nonlinear strongly monotone Lipschitz continuous operators. A successive iteration technique to solve the problem numerically is proposed, and its convergence is established. A variant of the Augmented Lagrangian, the so-called Alternating Direction Method of multipliers (ADMM), is used to decompose the original problem into two sub-problems, solve them sequentially and update the dual variables at each iteration. The influence of the thermal boundary condition on the behavior of contact forces and electrical potential is shown through graphical illustrations. [ABSTRACT FROM AUTHOR]

We consider solving nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We assume for the objective that its evaluation, gradient, and Hessian are inaccessible, while one can compute their stochastic estimates by, for example, subsampling. We propose a stochastic algorithm based on sequential quadratic programming (SQP) that uses a differentiable exact augmented Lagrangian as the merit function. To motivate our algorithm design, we first revisit and simplify an old SQP method Lucidi (J. Optim. Theory Appl. 67(2): 227–245, 1990) developed for solving deterministic problems, which serves as the skeleton of our stochastic algorithm. Based on the simplified deterministic algorithm, we then propose a non-adaptive SQP for dealing with stochastic objective, where the gradient and Hessian are replaced by stochastic estimates but the stepsizes are deterministic and prespecified. Finally, we incorporate a recent stochastic line search procedure Paquette and Scheinberg (SIAM J. Optim. 30(1): 349–376 2020) into the non-adaptive stochastic SQP to adaptively select the random stepsizes, which leads to an adaptive stochastic SQP. The global "almost sure" convergence for both non-adaptive and adaptive SQP methods is established. Numerical experiments on nonlinear problems in CUTEst test set demonstrate the superiority of the adaptive algorithm. [ABSTRACT FROM AUTHOR]

A mixed signal with several unknown modes is common in the industry and is hard to decompose. Variational Mode Decomposition (VMD) was proposed to decompose a signal into several amplitude-modulated modes in 2014, which overcame the limitations of Empirical Mode Decomposition (EMD), such as sensitivity to noise and sampling. We propose an improved VMD, which is simplified as iVMD. In the new algorithm, we further study and improve the mathematical model of VMD to adapt to the decomposition of the broad-band modes. In the new model, the ideal flattest response is applied, which is derived from the mathematical integral form and obtained from different-order derivatives of the improved modes' definitions. The harmonics can be treated via synthesis in our new model. The iVMD algorithm can decompose the complex harmonic signal and the broad-band modes. The new model is optimized with the alternate direction method of multipliers, and the modes with adaptive broad-band and their respective center frequencies can be decomposed. the experimental results show that iVMD is an effective algorithm based on the artificial and real data collected in our experiments. [ABSTRACT FROM AUTHOR]

We present a new algorithm for infinite-dimensional optimization with general constraints, called ALESQP. In short, ALESQP is an augmented Lagrangian method that penalizes inequality constraints and solves equality-constrained nonlinear optimization subproblems at every iteration. The subproblems are solved using a matrix-free trust-region sequential quadratic programming (SQP) method that takes advantage of iterative, i.e., inexact linear solvers, and is suitable for large-scale applications. A key feature of ALESQP is a constraint decomposition strategy that allows it to exploit problem-specific variable scalings and inner products. We analyze convergence of ALESQP under different assumptions. We show that strong accumulation points are stationary. Consequently, in finite dimensions ALESQP converges to a stationary point. In infinite dimensions we establish that weak accumulation points are feasible in many practical situations. Under additional assumptions we show that weak accumulation points are stationary. We present several infinite-dimensional examples where ALESQP shows remarkable discretization-independent performance in all of its iterative components, requiring a modest number of iterations to meet constraint tolerances at the level of machine precision. Also, we demonstrate a fully matrix-free solution of an infinite-dimensional problem with nonlinear inequality constraints. [ABSTRACT FROM AUTHOR]

The main purpose of this paper is the numerical analysis of a class of mathematical models that describe the contact between a thermo-piezoelectric body and a conductive foundation. Under the assumption of a static process, the material's behavior is modeled with a linear thermo-electro-elastic constitutive law and the frictional contact with Signorini's and Tresca's laws. A variational problem is derived and the existence of a unique weak solution is proved by combining arguments from the theory of variational inequalities with linear strongly monotone Lipschitz continuous operators. A successive iteration technique to linearize the problem by transforming it into an incremental recursive form is proposed, and its convergence is established. An Augmented Lagrangian variant, known as the Alternating Direction Multiplier Method (ADMM), is employed to split the original problem into two subproblems, resolve them sequentially, as well as update the dual variables at each iteration. To illustrate the performance of the proposed approach, several numerical simulations on two-dimensional test problems are carried out. [ABSTRACT FROM AUTHOR]

Owing to absorption, reflection, diffraction and, deplorable conditions, the capturing of underwater images present more challenges for the consumer. The proposed work focuses on Enhanced Augmented Lagrangian for image deblurring with additional performance-enhancing optical and sparse derivative prior to model the underwater imaging. The proposed method using Augmented Lagragian with Optical and Sparse Derivative prior is novel in the following ways: (i) usage of optical prior modeled after underwater imaging conditions, taking into account deformations and distortions that accompany the capture of underwater images (ii) sparse derivative prior helps to optimize fast convergence through regularization and faster computation with better edge preservation; (iii) the deblurring begins with the sparsest derivative beforehand and the final deblurred result achieving good dB improvements through sparse regulation. The weights with penalty and regularization ensure that each iteration through steep descent reaches a global minimum. The proposed algorithm performs better as compared to state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

The Boussinesq problem, which describes quasi-static indentation of a rigid punch into a deformable body, is studied within the context of nonlinear constitutive equations. By this, the material response expresses the linearized strain in terms of the stress and cannot be inverted in general. A contact area between the punch and the body is unknown a priori, whereas the total contact force is prescribed and yields a non-local integral condition. Consequently, the unilateral indentation problem is stated as a quasi-variational inequality for unknown variables of displacement, stress and indentation depth. The Lagrange multiplier approach is applied in order to establish well-posedness to the underlying physically and geometrically nonlinear problem based on augmented penalty regularization and applying the minimax theorem of Ekeland and Témam. A sufficient solvability condition implies response functions that are bounded, hemi-continuous, coercive and obey a convex potential. A typical example is power-law hardening models for titanium alloys, Norton–Hoff and Ramberg–Osgood materials. This article is part of the theme issue 'Non-smooth variational problems and applications'. [ABSTRACT FROM AUTHOR]

MR-STAT is an emerging quantitative magnetic resonance imaging technique which aims at obtaining multi-parametric tissue parameter maps from single short scans. It describes the relationship between the spatial-domain tissue parameters and the time-domain measured signal by using a comprehensive, volumetric forward model. The MR-STAT reconstruction solves a large-scale nonlinear problem, thus is very computationally challenging. In previous work, MR-STAT reconstruction using Cartesian readout data was accelerated by approximating the Hessian matrix with sparse, banded blocks, and can be done on high performance CPU clusters with tens of minutes. In the current work, we propose an accelerated Cartesian MR-STAT algorithm incorporating two different strategies: firstly, a neural network is trained as a fast surrogate to learn the magnetization signal not only in the full time-domain but also in the compressed low-rank domain; secondly, based on the surrogate model, the Cartesian MR-STAT problem is re-formulated and split into smaller sub-problems by the alternating direction method of multipliers. The proposed method substantially reduces the computational requirements for runtime and memory. Simulated and in-vivo balanced MR-STAT experiments show similar reconstruction results using the proposed algorithm compared to the previous sparse Hessian method, and the reconstruction times are at least 40 times shorter. Incorporating sensitivity encoding and regularization terms is straightforward, and allows for better image quality with a negligible increase in reconstruction time. The proposed algorithm could reconstruct both balanced and gradient-spoiled in-vivo data within 3 minutes on a desktop PC, and could thereby facilitate the translation of MR-STAT in clinical settings. [ABSTRACT FROM AUTHOR]

The present study introduces a formulation for topology optimization of structures with constraints on the first principal stress. We solve the problem considering local stress constraints via the augmented Lagrangian method, which enables the solution of large-scale problems without the need for ad hoc aggregation schemes and clustering methods. Numerical examples are provided which demonstrate the effectiveness of the framework for practical problems with numerous (e.g., in the range of million(s)) local constraints imposed on the maximum principal stress. One of the examples is a three-dimensional antenna support bracket, which represents a realistic engineering design problem. This example, which has more than one million constraints, is proposed as a benchmark problem for stress-constrained topology optimization. [ABSTRACT FROM AUTHOR]

A distributed method for concurrent traffic signal and routing control of traffic networks is proposed. The method is based on the multi-commodity store-and-forward model, in which the destinations are the commodities. The system benefits from the communication between vehicles and infrastructure, providing optimal signal timings to intersections and routes to vehicles on a link-by-link basis. Using the augmented Lagrangian to model the constraints into the objective, the baseline centralized problem is decomposed into a set of objective-coupled subproblems, one for each intersection, enabling the solution to be computed by a distributed-gradient projection algorithm. The intersection agents only need to communicate and coordinate with neighboring intersections to ensure convergence to the optimal solution while tolerating suboptimal iterations that offer more flexibility, unlike other distributed approaches. Through microsimulation, we demonstrate the effectiveness of the proposed algorithm in traffic networks with time-varying demand. Computational analysis shows that the distributed problem is suitable for real-time applications. A robustness analysis show that the distributed formulation enables a graceful degradation of the system in case of failure. • A multi-commodity traffic flow model cast into a linear dynamic network. • A distributed algorithm for linear dynamic networks and in particular applicable to multi-commodity urban signal traffic control and routing in a connected vehicle environment. • Demonstration of the real-time applicability and robustness to failure of the proposed algorithm and model. • Comparison through microscopic simulation with the corresponding centralized approach, with a decentralized backpressure algorithm, and with fixed-time. [ABSTRACT FROM AUTHOR]

The magnetohydrodynamics (MHD) equations are generally known to be difficult to solve numerically, due to their highly nonlinear structure and the strong coupling between the electromagnetic and hydrodynamic variables, especially for high Reynolds and coupling numbers. In this work, we present a scalable augmented Lagrangian preconditioner for a finite element discretization of the B-E formulation of the incompressible viscoresistive MHD equations. For stationary problems, our solver achieves robust performance with respect to the Reynolds and coupling numbers in two dimensions and good results in three dimensions. We extend our method to fully implicit methods for time-dependent problems which we solve robustly in both two and three dimensions. Our approach relies on specialized parameter-robust multigrid methods for the hydrodynamic and electromagnetic blocks. The scheme ensures exactly divergence-free approximations of both the velocity and the magnetic field up to solver tolerances. We confirm the robustness of our solver by numerical experiments in which we consider fluid and magnetic Reynolds numbers and coupling numbers up to 10,000 for stationary problems and up to 100,000 for transient problems in two and three dimensions. [ABSTRACT FROM AUTHOR]

We introduce a perturbed augmented Lagrangian method framework, which is a convenient tool for local analyses of convergence and rates of convergence of some modifications of the classical augmented Lagrangian algorithm. One example to which our development applies is the proximal augmented Lagrangian method. Previous results for this version required twice differentiability of the problem data, the linear independence constraint qualification, strict complementarity, and second-order sufficiency; or the linear independence constraint qualification and strong second-order sufficiency. We obtain a set of convergence properties under significantly weaker assumptions: once (not twice) differentiability of the problem data, uniqueness of the Lagrange multiplier, and second-order sufficiency (no linear independence constraint qualification and no strict complementarity); or even second-order sufficiency only. Another version to which the general framework applies is the smoothed augmented Lagrangian method, where the plus-function associated with penalization of inequality constraints is approximated by a family of smooth functions (so that the subproblems are twice differentiable if the problem data are). Furthermore, for all the modifications, inexact solution of subproblems is handled naturally. The presented framework also subsumes the basic augmented Lagrangian method, both exact and inexact. [ABSTRACT FROM AUTHOR]

In this paper, we propose in a Hilbertian setting a second-order time-continuous dynamic system with fast convergence guarantees to solve structured convex minimization problems with an affine constraint. The system is associated with the augmented Lagrangian formulation of the minimization problem. The corresponding dynamics brings into play three general time-varying parameters, each with specific properties, and which are, respectively, associated with viscous damping, extrapolation and temporal scaling. By appropriately adjusting these parameters, we develop a Lyapunov analysis which provides fast convergence properties of the values and of the feasibility gap. These results will naturally pave the way for developing corresponding accelerated ADMM algorithms, obtained by temporal discretization. [ABSTRACT FROM AUTHOR]

In remote sensing applications and medical imaging, one of the key points is the acquisition, real-time preprocessing and storage of information. Due to the large amount of information present in the form of images or videos, compression of these data is necessary. Compressed sensing is an efficient technique to meet this challenge. It consists in acquiring a signal, assuming that it can have a sparse representation, by using a minimum number of nonadaptive linear measurements. After this compressed sensing process, a reconstruction of the original signal must be performed at the receiver. Reconstruction techniques are often unable to preserve the texture of the image and tend to smooth out its details. To overcome this problem, we propose, in this work, a compressed sensing reconstruction method that combines the total variation regularization and the non-local self-similarity constraint. The optimization of this method is performed by using an augmented Lagrangian that avoids the difficult problem of nonlinearity and nondifferentiability of the regularization terms. The proposed algorithm, called denoising-compressed sensing by regularization (DCSR) terms, will not only perform image reconstruction but also denoising. To evaluate the performance of the proposed algorithm, we compare its performance with state-of-the-art methods, such as Nesterov's algorithm, group-based sparse representation and wavelet-based methods, in terms of denoising and preservation of edges, texture and image details, as well as from the point of view of computational complexity. Our approach permits a gain up to 25% in terms of denoising efficiency and visual quality using two metrics: peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). [ABSTRACT FROM AUTHOR]

We introduce exponential augmented Lagrangian methods for solving equilibrium problems in finite-dimensional spaces, extending the so-called quadratic augmented Lagrangian methods. Unlike the quadratic augmented Lagrangian methods that are at most first-order differentiable, our exponential augmented Lagrangian methods can be differentiable at any order. Therefore, second-order methods, such as Newton's methods, can be used to solve the subproblems generated by the augmented Lagrangian methods. We also present numerical results obtained based on implementing our proposed algorithms in Matlab. [ABSTRACT FROM AUTHOR]

In this paper, we aim at unifying, simplifying, and improving the convergence rate analysis of Lagrangian-based methods for convex optimization problems. We first introduce the notion of nice primal algorithmic map, which plays a central role in the unification and in the simplification of the analysis of most Lagrangian-based methods. Equipped with a nice primal algorithmic map, we then introduce a versatile generic scheme, which allows for the design and analysis of faster Lagrangian (FLAG) methods with new provably sublinear rate of convergence expressed in terms of function values and feasibility violation of the original (nonergodic) generated sequence. To demonstrate the power and versatility of our approach and results, we show that most well-known iconic Lagrangian-based schemes admit a nice primal algorithmic map and hence share the new faster rate of convergence results within their corresponding FLAG. [ABSTRACT FROM AUTHOR]

It is well-known that the primal quadratic growth condition of the classical augmented Lagrangian around a local minimizer can be obtained under the second-order sufficient optimality condition. In this paper, we show that those conditions are indeed equivalent. Moreover, we prove that the primal quadratic growth condition of the sharp augmented Lagrangian around a local minimizer is in fact equivalent to the weak second-order sufficient optimality condition. In addition, we present some secondary results involving the sharp augmented Lagrangian. [ABSTRACT FROM AUTHOR]

In this paper, the problem of load uncertainty in compliance problems is addressed where the uncertainty is described in the form of a set of finitely many loading scenarios. Computationally, more efficient methods are proposed to exactly evaluate and differentiate: (1) the mean compliance or (2) any scalar-valued function of the individual load compliances such as the weighted sum of the mean and standard deviation. The computational time complexities of all the proposed algorithms are analyzed, compared with the naive approaches and then experimentally verified. Finally, a mean compliance minimization problem, a risk-averse compliance minimization problem, and a maximum compliance-constrained problem are solved to showcase the efficacy of the proposed algorithms. The maximum compliance-constrained problem is solved using the augmented Lagrangian method and the method proposed for handling scalar-valued functions of the load compliances, where the scalar-valued function is the augmented Lagrangian function. [ABSTRACT FROM AUTHOR]

We present an augmented Lagrangian-based approach for stress-constrained topology optimization of structures subjected to general dynamic loading. The approach renders structures that satisfy the stress constraints locally at every time step. To solve problems with a large number of stress constraints, we normalize the penalty term of the augmented Lagrangian function with respect to the total number of constraints (i.e., the number of elements in the mesh times the number of time steps). Moreover, we solve the stress-constrained problem effectively by penalizing constraints associated with high stress values more severely than those associated with low stress values. We integrate the equations of motion using the HHT-α method and conduct the sensitivity analysis consistently with this method via the "discretize-then-differentiate" approach. We present several numerical examples that elucidate the effectiveness of the approach to solve dynamic, stress-constrained problems under several loading scenarios including loads that change in magnitude and/or direction and loads that change in position as a function of time. [ABSTRACT FROM AUTHOR]

Handling stress constraints is an important topic in topology optimization. In this paper, we introduce an interpretation of stresses as optimization variables, leading to an augmented Lagrangian formulation. This formulation takes two sets of optimization variables, i.e., an auxiliary stress variable per element, in addition to a density variable as in conventional density-based approaches. The auxiliary stress is related to the actual stress (i.e., computed by its definition) by an equality constraint. When the equality constraint is strictly satisfied, an upper bound imposed on the auxiliary stress design variable equivalently applies to the actual stress. The equality constraint is incorporated into the objective function as linear and quadratic terms using an augmented Lagrangian form. We further show that this formulation is separable regarding its two sets of variables. This gives rise to an efficient augmented Lagrangian solver known as the alternating direction method of multipliers (ADMM). In each iteration, the density variables, auxiliary stress variables, and Lagrange multipliers are alternatingly updated. The introduction of auxiliary stress variables enlarges the search space. We demonstrate the effectiveness and efficiency of the proposed formulation and solution strategy using simple truss examples and a dozen of continuum structure optimization settings. [ABSTRACT FROM AUTHOR]

In quantum embedding theories, a quantum many-body system is divided into localized clusters of sites which are treated with an accurate 'high-level' theory and glued together self-consistently by a less accurate 'low-level' theory at the global scale. The recently introduced variational embedding approach for quantum many-body problems combines the insights of semidefinite relaxation and quantum embedding theory to provide a lower bound on the ground-state energy that improves as the cluster size is increased. The variational embedding method is formulated as a semidefinite program (SDP), which can suffer from poor computational scaling when treated with black-box solvers. We exploit the interpretation of this SDP as an embedding method to develop an algorithm which alternates parallelizable local updates of the high-level quantities with updates that enforce the low-level global constraints. Moreover, we show how translation invariance in lattice systems can be exploited to reduce the complexity of projecting a key matrix to the positive semidefinite cone. • Scalable algorithm for variational quantum embedding, a semidefinite relaxation of the ground-state eigenvalue problem. • Alternates the solution of parallelizable local effective subproblems with global dual update steps. • Achieves convergence in number of iterations independent of cluster and system sizes. • Exploits translation invariance to efficiently project global matrix to the semidefinite cone. [ABSTRACT FROM AUTHOR]

We propose a robust and efficient augmented Lagrangian-type preconditioner for solving linearizations of the Oseen–Frank model arising in nematic and cholesteric liquid crystals. By applying the augmented Lagrangian method, the Schur complement of the director block can be better approximated by the weighted mass matrix of the Lagrange multiplier, at the cost of making the augmented director block harder to solve. In order to solve the augmented director block, we develop a robust multigrid algorithm which includes an additive Schwarz relaxation that captures a pointwise version of the kernel of the semi-definite term. Furthermore, we prove that the augmented Lagrangian term improves the discrete enforcement of the unit-length constraint. Numerical experiments verify the efficiency of the algorithm and its robustness with respect to problem-related parameters (Frank constants and cholesteric pitch) and the mesh size. [ABSTRACT FROM AUTHOR]

A reformulation of cardinality-constrained optimization problems into continuous nonlinear optimization problems with an orthogonality-type constraint has gained some popularity during the last few years. Due to the special structure of the constraints, the reformulation violates many standard assumptions and therefore is often solved using specialized algorithms. In contrast to this, we investigate the viability of using a standard safeguarded multiplier penalty method without any problem-tailored modifications to solve the reformulated problem. We prove global convergence towards an (essentially strongly) stationary point under a suitable problem-tailored quasinormality constraint qualification. Numerical experiments illustrating the performance of the method in comparison to regularization-based approaches are provided. [ABSTRACT FROM AUTHOR]

We develop a new computational model to describe hydro-mechanical coupling in fractured rocks composed of a linear poroelastic Biot medium and nonlinear elastic joints with constitutive response governed by the Barton–Bandis (BB) law. The model aims at capturing increase in stiffness induced by fracture closure during fluid withdrawal. The nonlinear hydro-mechanical formulation is constructed within the framework of the Discrete Fracture Model, with flow and geomechanical sub-systems coupled through a sequential iterative algorithm. The internal contact constraint arising from non-overlapping between opposite fracture faces is enforced through the weak fulfillment of the BB-law. Such a constraint is captured within the framework of the Augmented Lagrangian formulation, where the non-linear mechanical interaction is enforced adopting successive approximations of the Lagrange multiplier, interpreted as the contact pressure in the joint, supplemented by a penalty component associated with the rock stiffness. Furthermore, adopting a traditional flow based upscaling method, macroscopic permeabilities are numerically reconstructed with magnitude strongly dependent on the local stress state. Such a mechanical dependence of the homogenized properties is represented in a discrete manner through pseudo-coupling tables, with enormous potential to be explored within a preprocessing stage in reservoir simulators to compute multipliers relative to a chosen pre-stressed reference state, where input data is available. Numerical simulations are performed for some fracture arrangements illustrating the potential of the formulation proposed herein in bridging hydromechanical coupling at different scales in jointed rocks. [ABSTRACT FROM AUTHOR]

We present PolyStress, a Matlab implementation for topology optimization with local stress constraints considering linear and material nonlinear problems. The implementation of PolyStress is built upon PolyTop, an educational code for compliance minimization on unstructured polygonal finite elements. To solve the nonlinear elasticity problem, we implement a Newton-Raphson scheme, which can handle nonlinear material models with a given strain energy density function. To solve the stress-constrained problem, we adopt a scheme based on the augmented Lagrangian method, which treats the problem consistently with the local definition of stress without employing traditional constraint aggregation techniques. The paper discusses several theoretical aspects of the stress-constrained problem, including details of the augmented Lagrangian-based approach implemented herein. In addition, the paper presents details of the Matlab implementation of PolyStress, which is provided as electronic supplementary material. We present several numerical examples to demonstrate the capabilities of PolyStress to solve stress-constrained topology optimization problems and to illustrate its modularity to accommodate any nonlinear material model. Six appendices supplement the paper. In particular, the first appendix presents a library of benchmark examples, which are described in detail and can be explored beyond the scope of the present work. [ABSTRACT FROM AUTHOR]

A Uzawa block relaxation method, based on augmented Lagrangian functional and adaptive rule, is designed and analysed for free boundary problems with unilateral obstacle. We introduce an auxiliary unknown and augmented Lagrangian functional to transform the problem into a saddle-point problem, which can be solved by the Uzawa block relaxation method, and each iterative step consists of a linear problem while the auxiliary unknown is computed explicitly. The convergence speed of the method depends on the parameter heavily, and it is difficult to choose a proper parameter for individual problems. To improve the efficiency of the method, we propose an adaptive rule which adjusts the parameter automatically per iteration. Numerical examples show the performance of the proposed method for 1D and 2D free boundary problems. [ABSTRACT FROM AUTHOR]

The Boussinesq problem describing indentation of a rigid punch of arbitrary shape into a deformable solid body is studied within the context of a linear viscoelastic model. Due to the presence of a non-local integral constraint prescribing the total contact force, the unilateral indentation problem is formulated in the general form as a quasi-variational inequality with unknown indentation depth, and the Lagrange multiplier approach is applied to establish its well-posedness. The linear viscoelastic model that is considered assumes that the linearized strain is expressed by a material response function of the stress involving a Volterra convolution operator, thus the constitutive relation is not invertible. Since viscoelastic indentation problems may not be solvable in general, under the assumption of monotonically non-increasing contact area, the solution for linear viscoelasticity is constructed using the convolution for an increment of solutions from linearized elasticity. For the axisymmetric indentation of the viscoelastic half-space by a cone, based on the Papkovich–Neuber representation and Fourier–Bessel transform, a closed form analytical solution is constructed, which describes indentation testing within the holding-unloading phase. [ABSTRACT FROM AUTHOR]

The alternating direction method of multipliers (ADMM) is a classical effective method for solving two-block convex optimization subject to linear constraints. However, its convergence may not be guaranteed for multiple-block case without additional assumptions. One remedy might be the block-wise ADMM (BADMM), in which the variables are regrouped into two groups firstly and then the augmented Lagrangian function is minimized w.r.t. each block variable by the following scheme: using a Gauss–Seidel fashion to update the variables between each group, while using a Jacobi fashion to update the variables within each group. In order to derive its convergence property, a special proximal term is added to each subproblem. In this paper, we propose a new partial PPA block-wise ADMM where we only need to add proximal terms to the subproblems in the first group. At the end of each iteration, an extension step on all variables is performed with a fixed step size. As the subproblems in the second group are unmodified, the resulting sequence might yield better quality as well as potentially faster convergence speed. Preliminary experimental results show that the new algorithm is empirically effective on solving both synthetic and real problems when it is compared with several very efficient ADMM-based algorithms. [ABSTRACT FROM AUTHOR]

We consider two static problems which describe the contact between a piezoelectric body and an obstacle, the so-called foundation. The constitutive relation of the material is assumed to be electro-elastic and involves the nonlinear elastic constitutive Hencky's law. In the first problem, the contact is assumed to be frictionless, and the foundation is nonconductive, while in the second it is supposed to be frictional, and the foundation is electrically conductive. The contact is modeled with the normal compliance condition with finite penetration, the regularized Coulomb law, and the regularized electrical conductivity condition. The existence and uniqueness results are provided using the theory of variational inequalities and Schauder's fixed-point theorem. We also prove that the solution of the latter problem converges towards that of the former as the friction and electrical conductivity coefficients converge towards zero. The numerical solutions of the problems are achieved by using a successive iteration technique; their convergence is also established. The numerical treatment of the contact condition is realized using an Augmented Lagrangian type formulation that leads us to use Uzawa type algorithms. Numerical experiments are performed to show that the numerical results are consistent with the theoretical analysis. [ABSTRACT FROM AUTHOR]

Digital volume correlation (DVC), the volumetric extension of the popular digital image correlation (DIC) technique, is a powerful experimental tool for measuring 3D volumetric full-field displacements and strains. Most current DVC algorithms can be categorized into either local or finite-element-based global methods. As with most experimental approaches, there are drawbacks with each of these methods. In the local method the subvolume deformations are estimated independently and the computed displacement field may not necessarily be kinematically compatible. Thus, the deformation gradients can be noisy, especially when using small volumetric subsets. Although the global method often enforces kinematic compatibility, it generally incurs substantially greater computational costs than its local counterpart, which is especially significant for large volumetric data sets. To address these shortcomings, we present a new hybrid DVC algorithm, called augmented Lagrangian digital volume correlation (ALDVC), which combines the advantages of both the local (fast computation time) and global (compatible displacement field) methods. This new algorithm builds on our recent work on the augmented Lagrangian digital image correlation (2D-ALDIC) technique and solves the general motion optimization problem by using the alternating direction method of multipliers (ADMM). We demonstrate that our ALDVC algorithm has high accuracy and precision while maintaining low computational cost, and is a significant improvement compared to current local and global DVC methods. ALDVC is a computationally efficient algorithm to measure 3D volumetric displacements and strains. An open-source Matlab implementation is freely available. [ABSTRACT FROM AUTHOR]

In this paper we propose a splitting scheme which hybridizes the generalized conditional gradient with a proximal step and which we call the CGALP algorithm for minimizing the sum of three proper convex and lower-semicontinuous functions in real Hilbert spaces. The minimization is subject to an affine constraint, that, in particular, allows one to deal with composite problems (a sum of more than three functions) in a separable way by the usual product space technique. While classical conditional gradient methods require Lipschitz continuity of the gradient of the differentiable part of the objective, CGALP needs only differentiability (on an appropriate subset) and hence circumvents the intricate question of Lipschitz continuity of gradients. For the two remaining functions in the objective, we do not require any additional regularity assumption. The second function, possibly nonsmooth, is assumed simple; i.e., the associated proximal mapping is easily computable. For the third function, again nonsmooth, we just assume that its domain is weakly compact and that a linearly perturbed minimization oracle is accessible. In particular, this last function can be chosen to be the indicator of a nonempty bounded closed convex set in order to deal with additional constraints. Finally, the affine constraint is addressed by the augmented Lagrangian approach. Our analysis is carried out for a wide choice of algorithm parameters satisfying so-called open loop rules. As main results, under mild conditions, we show asymptotic feasibility with respect to the affine constraint, weak convergence of the dual multipliers, and convergence of the Lagrangian values to the saddle-point optimal value. We also provide pointwise and ergodic rates of convergence for both the feasibility gap and the Lagrangian values. [ABSTRACT FROM AUTHOR]

This paper presents a consistent topology optimization formulation for mass minimization with local stress constraints by means of the augmented Lagrangian method. To solve problems with a large number of constraints in an effective way, we modify both the penalty and objective function terms of the augmented Lagrangian function. The modification of the penalty term leads to consistent solutions under mesh refinement and that of the objective function term drives the mass minimization towards black and white solutions. In addition, we introduce a piecewise vanishing constraint, which leads to results that outperform those obtained using relaxed stress constraints. Although maintaining the local nature of stress requires a large number of stress constraints, the formulation presented here requires only one adjoint vector, which results in an efficient sensitivity evaluation. Several 2D and 3D topology optimization problems, each with a large number of local stress constraints, are provided. [ABSTRACT FROM AUTHOR]

In the context of numerical constrained optimization, we investigate stochastic algorithms, in particular evolution strategies, handling constraints via augmented Lagrangian approaches. In those approaches, the original constrained problem is turned into an unconstrained one and the function optimized is an augmented Lagrangian whose parameters are adapted during the optimization. The use of an augmented Lagrangian however breaks a central invariance property of evolution strategies, namely invariance to strictly increasing transformations of the objective function. We formalize nevertheless that an evolution strategy with augmented Lagrangian constraint handling should preserve invariance to strictly increasing affine transformations of the objective function and the scaling of the constraints—a subclass of strictly increasing transformations. We show that this invariance property is important for the linear convergence of these algorithms and show how both properties are connected. [ABSTRACT FROM AUTHOR]

A Multiple-Shooting Differential Dynamic Programming Algorithm is applied to a variety of constrained nonlinear optimal control problems, including classic benchmark problems, as well as a robotic arm problem and sensitive spacecraft trajectory optimization problems. The multiple-shooting extension presented in Part 1 is validated, as well as the Powell, Hestenes, and Rockafellar approach used in the treatment of equality and inequality path and terminal constraints. The results for example applications demonstrate the applicability of the algorithm to a variety of trajectory optimization problems. The advantages of the multiple-shooting approach over the single-shooting algorithm is evident in problems with high sensitivity. In particular, convergence of the multiple-shooting algorithm is demonstrated for complex spacecraft trajectory problems that are intractable using the single-shooting formulation. • The multiple-shooting differential dynamic programming algorithm is validated. • The multiple-shooting approach is effective for general optimal control problems. • The sensitivities of each subproblem are reduced with multiple-shooting. • The new algorithm can be used to solve complex and sensitive problems robustly. • The Augmented Lagrangian approach to inequality constraints is effective. [ABSTRACT FROM AUTHOR]

At each iteration of the augmented Lagrangian algorithm, a nonlinear subproblem is being solved. The number of inner iterations (of some/any method) needed to obtain a solution of the subproblem, or even a suitable approximate stationary point, is in principle unknown. In this paper we show that to compute an approximate stationary point sufficient to guarantee local superlinear convergence of the augmented Lagrangian iterations, it is enough to solve two quadratic programming problems (or two linear systems in the equality-constrained case). In other words, two inner Newtonian iterations are sufficient. To the best of our knowledge, such results are not available even under the strongest assumptions (of second-order sufficiency, strict complementarity, and the linear independence constraint qualification). Our analysis is performed under second-order sufficiency only, which is the weakest assumption for obtaining local convergence and rate of convergence of outer iterations of the augmented Lagrangian algorithm. The structure of the quadratic problems in question is related to the stabilized sequential quadratic programming and to second-order corrections. [ABSTRACT FROM AUTHOR]

We consider the problem of minimizing a smooth nonconvex function over a structured convex feasible set, that is, defined by two sets of constraints that are easy to treat when considered separately. In order to exploit the structure of the problem, we define an equivalent formulation by duplicating the variables and we consider the augmented Lagrangian of this latter formulation. Following the idea of the Alternating Direction Method of Multipliers (ADMM), we propose an algorithm where a two-blocks decomposition method is embedded within an augmented Lagrangian framework. The peculiarities of the proposed algorithm are the following: (1) the computation of the exact solution of a possibly nonconvex subproblem is not required; (2) the penalty parameter is iteratively updated once an approximated stationary point of the augmented Lagrangian is determined. Global convergence results are stated under mild assumptions and without requiring convexity of the objective function. Although the primary aim of the paper is theoretical, we perform numerical experiments on a nonconvex problem arising in machine learning, and the obtained results show the practical advantages of the proposed approach with respect to classical ADMM. [ABSTRACT FROM AUTHOR]

An interesting, yet challenging problem in topology optimization consists of finding the lightest structure that is able to withstand a given set of applied loads without experiencing local material failure. Most studies consider material failure via the von Mises criterion, which is designed for ductile materials. To extend the range of applications to structures made of a variety of different materials, we introduce a unified yield function that is able to represent several classical failure criteria including von Mises, Drucker-Prager, Tresca, Mohr-Coulomb, Bresler-Pister and Willam- Warnke, and use it to solve topology optimization problems with local stress constraints. The unified yield function not only represents the classical criteria, but also provides a smooth representation of the Tresca and the Mohr-Coulomb criteria-an attribute that is desired when using gradient-based optimization algorithms. The present framework has been built so that it can be extended to failure criteria other than the ones addressed in this investigation. We present numerical examples to illustrate how the unified yield function can be used to obtain different designs, under prescribed loading or designdependent loading (e.g. self-weight), depending on the chosen failure criterion. [ABSTRACT FROM AUTHOR]

Multiple-shooting benefits a wide variety of optimal control algorithms by alleviating large sensitivities present in highly nonlinear problems, improving robustness to initial guesses, and increasing the potential for parallel implementation. In this paper series, motivated by the challenges of optimizing highly sensitive spacecraft trajectories, the multiple-shooting approach is embedded for the first time in the formulation of a Differential Dynamic Programming algorithm. Contrary to traditional nonlinear programming methods which necessitate minimal modification of the formulation in order to incorporate multiple-shooting principles, DDP requires novel non-trivial derivations, in order to include the initial conditions of the multiple-shooting subintervals and track their sensitivities. The initial conditions are updated in a necessary additional algorithmic step. A null-space trust-region method is used for the solution of the quadratic subproblems, and equality and inequality path and terminal constraints are handled through a general augmented Lagrangian approach. The propagation of the trajectory and the optimization of its controls are decoupled through the use of State-Transition Matrices. Part 1 of the paper series provides the necessary theoretical developments and implementation details for the algorithm. Part 2 contains validation and application cases. • The multiple-shooting framework is applied to Differential Dynamic Programming. • The necessary quadratic expansions feedback control laws are developed. • An Augmented Lagrangian approach is applied for equality and inequality constraints. [ABSTRACT FROM AUTHOR]