Search

Total variation (TV) is a one of the most popular regularizers for stabilizing the solution of ill-posed inverse problems. This paper proposes a novel proximal-gradient algorithm for minimizing TV regularized least-squares cost functionals. Unlike traditional methods that require nested iterations for computing the proximal step of TV, our algorithm approximates the latter with several simple proximals that have closed form solutions. We theoretically prove that the proposed parallel proximal method achieves the TV solution with arbitrarily high precision at a global rate of converge that is equivalent to the fast proximal-gradient methods. The results in this paper have the potential to enhance the applicability of TV for solving very large-scale imaging inverse problems. [ABSTRACT FROM AUTHOR]

This paper presents nonlinear optimization-based position and speed estimation scheme for IPMSM drives with arbitrary signal injection generated by inherent switching ripples associated with finite control set model predictive control (FCSMPC). The existing standard sensorless techniques are not suitable for FCSMPC which applies voltage vectors directly to an electrical machine without a modulator. The proposed method optimizes the nonlinear cost function derived from the standard IPMSM model with respect to position and speed at every sampling interval. This method can be applied to any type of signal injection and, hence, an ideal candidate for sensorless FCSMPC. In this method, the signal injection is needed only to generate persistent excitation to maintain the observability at low speeds. A strong persistent excitation is always present with FCSMPC except at standstill where the control applies null vector when the reference currents are zero. This situation is overcome in this paper by introducing a small negative $d$ -axis current at standstill. Thus, the proposed method can estimate the position and speed over a wide speed range starting from standstill to the rated speed without a changeover or additional signal injection. This paper also presents detailed convergence analysis and proposes a compensator for standstill operation that prevents converging to saddle and symmetrical solutions, and therefore, also eliminates the well-known ambiguity of $\pi$ rad in position estimation. The performance of the proposed sensorless scheme is experimentally verified for a wide range of operating conditions. [ABSTRACT FROM AUTHOR]

The spiral optimization (SPO) algorithm proposed by Tamura and Yasuda is a relatively novel and simple search concept inspired by natural spiral phenomena. This algorithm searches continuous space using no gradient and only spiral trajectories composed of spiral vectors generated by deterministic spiral models. The primary purpose of this paper is to propose conditions and settings that mathematically ensure the convergence of the SPO algorithm to a stationary point. The conditions relating to the sizes and directions of the spiral vectors and the initial search points are based on direct search theory and recent SPO algorithm theories. The presented convergence was numerically verified using test functions with different properties. [ABSTRACT FROM AUTHOR]

The distributed Volt/Var control solution is a promising one for active distribution networks (ADNs), but a critical issue is how to improve the distributed algorithm's convergence with less communication burden. In this paper, we propose a bi-level consensus alternation direction multiplier method-based fully distributed Volt/Var optimization algorithm (B-FADMM) for ADNs by exploiting inverter-based rapid control devices. In the first level, the Volt/Var optimal solution of subpartitions is obtained in parallel, and the second level completes the variable increment of each partition to correct the search direction of the first level. In the algorithm, firstly, an adaptive matrix $\omega $ initialization strategy is proposed to adapt to the different magnitude of variables, which avoids non-convergence or slow convergence of the algorithm. Secondly, the null-space method is used in the second level of the algorithm to obtain the null-space basis matrix on the active constraint of the subpartition, which reduces the dimensions of the problem and the computational complexity. In addition, the conjugate gradient (CG) algorithm is used to update the dual multiplier of each partition only by exchanging coupling information among partitions, which protects information privacy. Experiments showed the proposed approach attains the same result as the centralized, the computational performance of the proposed method is far superior to some popular other distributed methods in terms of accuracy, computational efficiency, and scalability. [ABSTRACT FROM AUTHOR]

This paper studies the convergence properties of the well-known message-passing algorithm for convex optimisation. Under the assumption of pairwise separability and scaled diagonal dominance, asymptotic convergence is established and a simple bound for the convergence rate is provided for message-passing. In comparison with previous results, our results do not require the given convex program to have known convex pairwise components and that our bound for the convergence rate is tighter and simpler. When specialised to quadratic optimisation, we generalise known results by providing a very simple bound for the convergence rate. [ABSTRACT FROM AUTHOR]

We consider a distributed stochastic optimization problem in networks with finite number of nodes. Each node adjusts its action to optimize the global utility of the network, which is defined as the sum of local utilities of all nodes. While Gradient descent method is a common technique to solve such optimization problem, the computation of the gradient may require much information exchange. In this paper, we consider that each node can only have a noisy numerical observation of its local utility, of which the closed-form expression is not available. This assumption is quite realistic, especially when the system is either too complex or constantly changing. Nodes may exchange partially the observation of their local utilities to estimate the global utility at each timeslot. We propose a distributed algorithm based on stochastic perturbation, under the assumption that each node has only part of the local utilities of the other nodes. We use stochastic approximation tools to prove that our algorithm converges almost surely to the optimum, given that the objective function is smooth and strictly concave. The convergence rate is also derived, under the additional assumption of strongly concave objective function. It is shown that the convergence rate scales as O (K−0.5) after a sufficient number of iterations K > K0, which is the optimal rate order in terms of K for our problem. Although the proposed algorithm can be applied to general optimization problems, we perform simulations for a typical power control problem in wireless networks and present numerical results to corroborate our claims. [ABSTRACT FROM AUTHOR]

With the increasing of the data size and the development of multi-core computers, asynchronous parallel stochastic optimization algorithms such as KroMagnon have gained significant attention. In this paper, we propose a new Sparse approximation and asynchronous parallel Stochastic Variance Reduced Gradient (SSVRG) method for sparse and high-dimensional machine learning problems. Unlike standard SVRG and its asynchronous parallel variant, KroMagnon, the snapshot point of SSVRG is set to the average of all the iterates in the previous epoch, which allows it to take much larger learning rates and also makes it more robust to the choice of learning rates. In particular, we use the sparse approximation of the popular SVRG estimator to perform completely sparse updates at all iterations. Therefore, SSVRG has a much lower per-iteration computational cost than its dense counterpart, SVRG++, and is very friendly to asynchronous parallel implementation. Moreover, we provide the convergence guarantees of SSVRG for both strongly convex and non-strongly convex problems, while existing asynchronous algorithms (e.g., KroMagnon and ASAGA) only have convergence guarantees for strongly convex problems. Finally, we extend SSVRG to non-smooth and asynchronous parallel settings. Numerical experimental results demonstrate that SSVRG converges significantly faster than the state-of-the-art asynchronous parallel methods, e.g., KroMagnon, and is usually more than three orders of magnitude faster than SVRG++. [ABSTRACT FROM AUTHOR]

Memory-based LT encoders (MBLTEs) have been shown to have better performance than the regular LT encoder in terms of bit error rate (BER) and decoding convergence speed. In this paper, we explore the entire family of MBLTEs for finite block-length codes over the binary erasure channel (BEC). We propose an algorithm to extend the first and second order MBLTE approach to an arbitrary $i$ -th order MBLTE. We analyze the performance of such encoders mathematically by characterizing the expected accumulated number of recovered variable nodes at each decoding round. We define the threshold of the memory-based encoding method (MBEM) and show that the performance of MBLTEs increases as the memory order increases up to the point where this threshold is achieved. Beyond this point, we show that the performance of MBLTEs saturates if the channel erasure probability is zero and degrades otherwise. We formulate an optimization problem to solve for the optimal memory order based on whether or not the MBEM threshold is achieved. We present an extensive set of numerical results. These show agreement between our analysis and computer simulations. They also show that our optimization problem is efficient in determining the optimal memory order of MBLTEs in terms of decoding convergence speed, BER/frame-error-rate, and error floor. [ABSTRACT FROM AUTHOR]

In this paper, we study the problem of learning a shallow artificial neural network that best fits a training data set. We study this problem in the over-parameterized regime where the numbers of observations are fewer than the number of parameters in the model. We show that with the quadratic activations, the optimization landscape of training, such shallow neural networks, has certain favorable characteristics that allow globally optimal models to be found efficiently using a variety of local search heuristics. This result holds for an arbitrary training data of input/output pairs. For differentiable activation functions, we also show that gradient descent, when suitably initialized, converges at a linear rate to a globally optimal model. This result focuses on a realizable model where the inputs are chosen i.i.d. from a Gaussian distribution and the labels are generated according to planted weight coefficients. [ABSTRACT FROM AUTHOR]

Optimal design of electric machines using finite-element analysis (FEA) necessitates too much computation time to maintain high accuracy. To reduce the computation time and obtain a global optimum in multi-modal problems, this paper presents differing extent mesh adaptive direct search (DEMADS). The proposed algorithm is classified into three groups according to the poll searching frame and mesh size for assigning a global and local searching. In particular, DEMADS shares information among solutions based on a parallel multi-start strategy for earlier stopping by deviation and checking revisits. The effectiveness of DEMADS has been validated and evaluated as the function call number and the convergence number at a global optimum using benchmarking function. Finally, it is applied to an optimization problem that minimize torque ripple of a spoke-type permanent magnet synchronous machine via FEA. [ABSTRACT FROM AUTHOR]

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem. [ABSTRACT FROM AUTHOR]

In this paper, we propose a cost-efficient beamforming optimization algorithm for multi-user wireless communication systems associated with the intelligent reflecting surface (IRS). From the baseline successive refinement algorithm, which gives a sub-optimal solution for the power minimization problem under the signal-to-interference-plus-noise-ratio (SINR) constraint at each user, several optimization techniques are proposed to reduce the computation complexity while maintaining the algorithm-level performance. To reduce the number of required multiply-accumulate (MAC) operations, we first simplify the complicated matrix inversion by utilizing the channel hardening effect. We also present the two-phase refinement process for the group-level optimization of phase-shift elements, further relaxing the computation complexity as well as the processing latency. Applying the proposed optimization techniques, as a result, numerical results show that the fully-optimized algorithm can reduce the computational costs by up to 89.4% while showing less than 1 dB power loss, leading to the practical solution for the next-generation IRS-aided communication. [ABSTRACT FROM AUTHOR]

This paper studies a class of distributed convex optimization problems by a set of agents in which each agent only has access to its own local convex objective function and the estimate of each agent is restricted to both coupling linear constraint and individual box constraints. Our focus is to devise a distributed primal-dual gradient algorithm for working out the problem over a sequence of time-varying general directed graphs. The communications among agents are assumed to be uniformly strongly connected. A column-stochastic mixing matrix and a fixed step-size are applied in the algorithm which exactly steers all the agents to asymptotically converge to a global optimal solution. Based on the standard strong convexity and the smoothness assumptions of the objective functions, we show that the distributed algorithm is capable of driving the whole network to geometrically converge to an optimal solution of the convex optimization problem only if the step-size does not exceed some upper bound. We also give an explicit analysis for the convergence rate of the proposed optimization algorithm. Simulations on economic dispatch problems and demand response problems in power systems are performed to illustrate the effectiveness of the proposed optimization algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes the design of gray wolf optimizer (GWO) algorithm-based fractional order proportional integral (FOPI) controller for power factor correction (PFC) using interleaved dc–dc single ended primary inductance converter converter with improved performance characteristics such as speed, accuracy, and stability. The detailed analysis and modeling of the proposed converter are presented. The optimal parameters of FOPI controller are obtained using GWO algorithm to improve the speed of response of the system. FOPI controller provides better performance in terms of robustness and stability since it has one more adjustable parameter than the conventional proportional integral (PI) controller. The cascade control strategy is implemented for the proposed converter with GWO-based FOPI controller as inner current and outer voltage controllers for PFC and load voltage regulation. The performance characteristics of the proposed system are analyzed and compared with conventional and other optimization techniques using MATLAB/Simulink software. A prototype of the proposed converter controlled by TMS320C2000 microcontroller for an output power of 200 W is tested and validated with the simulation results. The proposed system is used for powering devices like robots in critical applications such as military, space craft, etc., where stability, speed, and accuracy are major concerns. [ABSTRACT FROM AUTHOR]

Endmember extraction (EE) plays an important role in the quantitative analysis of hyperspectral images, as the main step in the decomposition of mixed pixels. At present, scholars have proposed many EE algorithms based on the linear spectral mixture model and the convex geometry principle, such as the pixel purity index (PPI) and the vertex component analysis (VCA). At the same time, many intelligent optimization algorithms, such as the particle swarm optimization (PSO) and the discrete PSO (DPSO), have been applied to EE, which can get promising results for real images. However, PSO and DPSO have theoretical limitations and cannot guarantee the global convergence. The problem of premature convergence will reduce the accuracy of the EE result. The quantum-behaved PSO (QPSO) can theoretically guarantee the convergence of the algorithm by combining the quantum mechanics into the PSO. In order to increase the accuracy of the algorithm, this paper proposes an improved QPSO (IQPSO) algorithm for EE. IQPSO has made innovations in population coding and initialization methods. Besides, the collaborative approach for updating the optimal positions of particles can help to solve the difficulties caused by high dimensions. The experimental results show that IQPSO can extract endmembers efficiently and effectively. [ABSTRACT FROM AUTHOR]

This paper proposes a novel approach to suppressing bounded exogenous disturbances in a linear discrete-time control system by a static state- or output-feedback control law. The approach is based on reducing the original problem to a nonconvex matrix optimization problem with the gain matrix as one variable. The latter problem is solved by the gradient method; its convergence is theoretically justified for several important special cases. An example is provided to demonstrate the effectiveness of the iterative procedure proposed. [ABSTRACT FROM AUTHOR]

In this paper, we propose two communication-efficient decentralized optimization algorithms over a general directed multi-agent network. The first algorithm, termed Compressed Push-Pull (CPP), combines the gradient tracking Push-Pull method with communication compression. We show that CPP is applicable to a general class of unbiased compression operators and achieves linear convergence rate for strongly convex and smooth objective functions. The second algorithm is a broadcast-like version of CPP (B-CPP), and it also achieves linear convergence rate under the same conditions on the objective functions. B-CPP can be applied in an asynchronous broadcast setting and further reduce communication costs compared to CPP. Numerical experiments complement the theoretical analysis and confirm the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

In this paper, a novel algorithm “firework optimization” (FO) is introduced, as well as its hybridization with generating set search (GSS). FO is an exploration algorithm, which intends to explore a vast region within the search space in order to determine regions that may contain a regional optimum. As the regions with potential are defined, GSS, the exploitation algorithm is applied to determine whether regions determined by FO contain a local or a global optimum. The validity of FO and GSS hybridization algorithm is verified using Branin and Six-Hump Camel test functions, and the hybridization algorithm is applied to an interior permanent magnet synchronous motor, in order to minimize torque ripple. [ABSTRACT FROM AUTHOR]

As nanometer technology advances, conventional optical proximity correction (OPC) that minimizes the edge placement error (EPE) at the nominal process condition alone often leads to poor process windows. To improve the mask printability across various process corners, process-window OPC optimizes EPE for multiple process corners, but often suffers long runtime, due to repeated lithographic simulations. This paper presents an efficient process variation (PV)-aware mask optimization framework, namely PVOPC, to simultaneously minimize EPE and PV band with fast convergence. The PVOPC framework includes EPE-sensitivity-driven dynamic fragmentation, PV-aware EPE modeling, and correction with three new EPE-converging techniques and a systematic subresolution-assisted feature insertion algorithm. Experimental results show that our approach efficiently achieves high-quality EPE and PV band results. [ABSTRACT FROM AUTHOR]

This paper investigates a wireless federated learning (FL) network with limited communication bandwidth, where multiple mobile clients train their individual models with the help of one central server. We consider the practical communication scenarios, where the clients should complete the local computation and model upload within a defined latency. By jointly exploiting the dynamic characteristics of wireless channels and computational capability at the clients, we optimize the federated learning network by maximizing the number of active clients under the constraints of both latency and bandwidth. Specifically, we propose two bandwidth allocation (BA) schemes, where scheme I is based on the instantaneous channel state information (CSI), while scheme II employs the particle swarm optimization (PSO) method, based on the statistical CSI. Simulation results on the test accuracy and convergence rate are finally provided to demonstrate the advantages of the proposed optimization schemes for the considered FL network. [ABSTRACT FROM AUTHOR]

While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approach, this paper presents a new (stochastic) distributed algorithm with Nesterov momentum for accelerated optimization of non-convex and non-smooth problems. Theoretically, we show that the proposed algorithm can achieve an $\epsilon$ -stationary solution under a constant step size with $\mathcal {O}(1/\epsilon ^2)$ computation complexity and $\mathcal {O}(1/\epsilon)$ communication complexity when the epigraph of the non-smooth term is a polyhedral set. When compared to the existing gradient tracking based methods, the proposed algorithm has the same order of computation complexity but lower order of communication complexity. To the best of our knowledge, the presented result is the first stochastic algorithm with the $\mathcal {O}(1/\epsilon)$ communication complexity for non-convex and non-smooth problems. Numerical experiments for a distributed non-convex regression problem and a deep neural network based classification problem are presented to illustrate the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

This paper introduces the stochastic Fejér-monotone hybrid steepest descent method (S-FM-HSDM) to solve affinely constrained and composite convex minimization tasks. The minimization task is not known exactly; noise contaminates the information about the composite loss function and the affine constraints. S-FM-HSDM generates sequences of random variables that, under certain conditions and with respect to a probability space, converge point-wise to solutions of the noiseless minimization task. S-FM-HSDM enjoys desirable attributes of optimization techniques such as splitting of variables and constant step size (learning rate). Furthermore, it provides a novel way of exploiting the information about the affine constraints via fixed-point sets of appropriate nonexpansive mappings. Among the offsprings of S-FM-HSDM, the hierarchical recursive least squares (HRLS) takes advantage of S-FM-HSDM's versatility toward affine constraints and offers a novel twist to LS by generating sequences of estimates that converge to solutions of a hierarchical optimization task: minimize a convex loss over the set of minimizers of the ensemble LS loss. Numerical tests on a sparsity-aware LS task show that HRLS compares favorably to several state-of-the-art convex, as well as non-convex, stochastic-approximation, and online-learning counterparts. [ABSTRACT FROM AUTHOR]

In this paper, we consider a nonlinear least squares model for the phase retrieval problem. Since the Hessian matrix may not be positive definite and the Gauss–Newton (GN) matrix is singular at any optimal solution, we propose a modified Levenberg–Marquardt (LM) method, where the Hessian is substituted by a summation of the GN matrix and a regularization term. Similar to the well-known Wirtinger flow (WF) algorithm under certain assumptions, we start from an initial point provably close to the set of the global optimal solutions. Global linear convergence and local quadratic convergence to the global solution set are established by estimating the smallest nonzero eigenvalues of the GN matrix, establishing local error bound properties and constructing a modified regularization condition. The computational cost becomes tractable if a preconditioned conjugate gradient method is applied to solve the LM equation inexactly. Specifically, the pre-conditioner is constructed from the expectation of the LM coefficient matrix by assuming the independence between the measurements and iteration point. The initialization can be dropped by slightly modifying the algorithm. Preliminary numerical experiments show that our algorithm is robust and it is often faster than the WF method on both random examples and natural image recovery. [ABSTRACT FROM AUTHOR]

In this two-part paper, we present a novel framework and methodology to analyze data from certain image-based biochemical assays, e.g., ELISPOT and Fluorospot assays. In this second part, we focus on our algorithmic contributions. We provide an algorithm for functional inverse diffusion that solves the variational problem we posed in Part I. As part of the derivation of this algorithm, we present the proximal operator for the non-negative group-sparsity regularizer, which is a novel result that is of interest in itself, also in comparison to previous results on the proximal operator of a sum of functions. We then present a discretized approximated implementation of our algorithm and evaluate it both in terms of operational cell-detection metrics and in terms of distributional optimal-transport metrics. [ABSTRACT FROM AUTHOR]

Distributed optimization in multi-agent systems under sparsity constraints has recently received a lot of attention. In this paper, we consider the in-network minimization of a continuously differentiable nonlinear function which is a combination of local agent objective functions subject to sparsity constraints on the variables. A crucial issue of in-network optimization is the handling of the communications, which may be expensive. This calls for efficient algorithms, that are able to reduce the number of required communication links and transmitted messages. To this end, we focus on asynchronous and randomized distributed techniques. Based on consensus techniques and iterative hard thresholding methods, we propose three methods that attempt to minimize the given function, promoting sparsity of the solution: asynchronous hard thresholding (AHT), broadcast hard thresholding (BHT), and gossip hard thresholding (GHT). Although similar in many aspects, it is difficult to obtain a unified analysis for the proposed algorithms. Specifically, we theoretically prove the convergence and characterize the limit points of AHT in regular networks under some proper assumptions on the functions to be minimized. For BHT and GHT, instead, we characterize the fixed points of the maps that rule their dynamics in terms of stationary points of original problem. Finally, we illustrate the implementation of our techniques in compressed sensing and present several numerical results on performance and number of transmissions required for convergence. [ABSTRACT FROM PUBLISHER]

Fast automatic image registration is an important prerequisite for image-guided clinical procedures. However, due to the large number of voxels in an image and the complexity of registration algorithms, this process is often very slow. Stochastic gradient descent is a powerful method to iteratively solve the registration problem, but relies for convergence on a proper selection of the optimization step size. This selection is difficult to perform manually, since it depends on the input data, similarity measure and transformation model. The Adaptive Stochastic Gradient Descent (ASGD) method is an automatic approach, but it comes at a high computational cost. In this paper, we propose a new computationally efficient method (fast ASGD) to automatically determine the step size for gradient descent methods, by considering the observed distribution of the voxel displacements between iterations. A relation between the step size and the expectation and variance of the observed distribution is derived. While ASGD has quadratic complexity with respect to the transformation parameters, fast ASGD only has linear complexity. Extensive validation has been performed on different datasets with different modalities, inter/intra subjects, different similarity measures and transformation models. For all experiments, we obtained similar accuracy as ASGD. Moreover, the estimation time of fast ASGD is reduced to a very small value, from 40 s to less than 1 s when the number of parameters is 105, almost 40 times faster. Depending on the registration settings, the total registration time is reduced by a factor of 2.5–7 \times for the experiments in this paper. [ABSTRACT FROM PUBLISHER]

This paper proposes a parallel multimodal optimization algorithm that is combined with electromagnetic transient simulation in a platform that unifies the setup, test, and execution of optimal designs for power systems. The algorithm speeds up the design of power systems as its computations can be executed independently on a highly parallelized environment. Additional speedup is achieved by using a surrogate model to estimate the objective function in regions of suspected local optima. The estimated functions can be used in the subsequent stages of postoptimization studies, such as sensitivity analyses. Comparative studies, in terms of computation time, are conducted against sequential execution of the proposed algorithm. The optimal design of a VSC-HVDC transmission is described to demonstrate the capabilities of the proposed algorithm. [ABSTRACT FROM PUBLISHER]

With the advancement in information technology, datasets with an enormous amount of data are available. The classification task on these datasets is more time- and memory-consuming as the number of data increases. The support vector machine (SVM), which is arguably the most popular classification technique, has disappointing performance in dealing with large datasets due to its constrained optimization problem. To deal with this challenge, the variant SVM (VSVM) has been utilized which has the fraction (1/2)b2 in its primal objective function, where b is the bias of the desired hyperplane. The VSVM has been solved with different optimization techniques in more time- and memory-efficient fashion. However, there is no guarantee that its optimal solution is the same as the standard SVM. In this paper, we introduce the generalized VSVM (GVSVM) which has the fraction (1/2)b2 in its primal objective function, for a fixed positive scalar t. Further, we present the thorough theoretical insights that indicate the optimal solution of the GVSVM tends to the optimal solution of the standard SVM as t → ∞. One vital corollary is to derive a closed-form formula to obtain the bias term in the standard SVM. Such a formula obviates the need of approximating it, which is the modus operandi to date. An efficient neural network is then proposed to solve the GVSVM dual problem, which is asymptotically stable in the sense of Lyapunov and converges globally exponentially to the exact solution of the GVSVM. The proposed neural network has less complexity in architecture and needs fewer computations in each iteration in comparison to the existing neural solutions. Experiments confirm the efficacy of the proposed recurrent neural network and the proximity of the GVSVM and the standard SVM solutions with more significant values of t. [ABSTRACT FROM AUTHOR]

Digital pre-distortion (DPD) systems are employed to reduce intermodulation at the output of intrinsic non-linear RF power amplifiers. Typical DPD algorithms use time-domain linear regression to calculate the model coefficients used to compensate the amplifier’s distortion. This solution has high complexity and limited performance. This paper presents an innovative DPD system where the feedback information is a scalar measurement taken from the feedback signal. Different feedback information can be obtained from the distorted signal, e.g., the adjacent channel leakage rejection or the spectral mask margin. The DPD model coefficients estimation becomes a generic numerical optimization problem and is conducted iteratively in a coefficient-by-coefficient basis. A new coefficient orthogonalization algorithm removes the interaction between orders, resulting in a faster convergence time and lower output power variation across the iterations. The numerical optimization problem is simplified, since the nonlinear model becomes orthogonal in the intermodulation domain, i.e., the adjustment of given coefficient does not affect the others. Simulations and experimental results show that the proposed algorithm can effectively and efficiently compensate the nonlinear distortion and reduce the spectral regrowth. [ABSTRACT FROM AUTHOR]

This paper reports on the feasibility of using a quasi-Newton optimization algorithm, limited-memory Broyden-Fletcher-Goldfarb-Shanno with boundary constraints (L-BFGS-B), for penalized image reconstruction problems in emission tomography (ET). For further acceleration, an additional preconditioning technique based on a diagonal approximation of the Hessian was introduced. The convergence rate of L-BFGS-B and the proposed preconditioned algorithm (L-BFGS-B-PC) was evaluated with simulated data with various factors, such as the noise level, penalty type, penalty strength and background level. Data of three 18F-FDG patient acquisitions were also reconstructed. Results showed that the proposed L-BFGS-B-PC outperforms L-BFGS-B in convergence rate for all simulated conditions and the patient data. Based on these results, L-BFGS-B-PC shows promise for clinical application. [ABSTRACT FROM PUBLISHER]

Distributed optimization for solving nonconvex optimal power flow (OPF) problems in power systems has attracted considerable attention in the last decade. Most studies are based on the geographical decomposition of the IEEE test systems for verifying the feasibility of the proposed approaches. However, it is not clear if one can extrapolate from these studies that similar approaches can also be applied to very large-scale real-world systems. In this paper, we show, for the first time, that distributed optimization can be effectively applied to a large-scale real transmission network, namely, the Polish 2383-bus system for which no predefined partitions exist, by using a recently developed partitioning technique. More specifically, the problem solved is the ac OPF problem with geographical decomposition of the network using the alternating direction method of multipliers (ADMM) method in conjunction with the partitioning technique. Through extensive simulations and analytical studies, we show that with the presented partitioning technique the convergence performance of the ADMM method can be improved substantially, which enables the application of distributed approaches to very large-scale systems. [ABSTRACT FROM PUBLISHER]

The emergence of big data has caused a dramatic shift in the operating regime for optimization algorithms. The performance bottleneck, which used to be computations, is now often communications. Several gradient compression techniques have been proposed to reduce the communication load at the price of a loss in solution accuracy. Recently, it has been shown how compression errors can be compensated for in the optimization algorithm to improve the solution accuracy. Even though convergence guarantees for error-compensated algorithms have been established, there is very limited theoretical support for quantifying the observed improvements in solution accuracy. In this paper, we show that Hessian-aided error compensation, unlike other existing schemes, avoids accumulation of compression errors on quadratic problems. We also present strong convergence guarantees of Hessian-based error compensation for stochastic gradient descent. Our numerical experiments highlight the benefits of Hessian-based error compensation, and demonstrate that similar convergence improvements are attained when only a diagonal Hessian approximation is used. [ABSTRACT FROM AUTHOR]

This paper proposes a hybrid, multiobjective optimization algorithm enabling global optimum tracking in permanent-magnet (PM) traction motor design. The methodology developed is based on the Artificial Bee Colony technique, strength Pareto evolutionary algorithm, and differential evolution strategy ensuring fast and reliable convergence to the optimal Pareto front. The effectiveness of the derived methodology is compared with other well-established and powerful algorithms from the literature through both appropriate test functions and an application example concerning an unequal teeth surface-mounted PM wheel motor design. [ABSTRACT FROM AUTHOR]

We consider the problem of recovering a complete (i.e., square and invertible) matrix A0 , from Y \in \mathbb R ^n \times p with Y = \boldsymbol A0 X0 , provided X0 is sufficiently sparse. This recovery problem is central to theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals and finds numerous applications in modern signal processing and machine learning. We give the first efficient algorithm that provably recovers A0 when X0 has O \left ( n \right ) nonzeros per column, under suitable probability model for X0 . Our algorithmic pipeline centers around solving a certain nonconvex optimization problem with a spherical constraint, and hence is naturally phrased in the language of manifold optimization. In a companion paper, we have showed that with high probability, our nonconvex formulation has no “spurious” local minimizers and around any saddle point, the objective function has a negative directional curvature. In this paper, we take advantage of the particular geometric structure and describe a Riemannian trust region algorithm that provably converges to a local minimizer with from arbitrary initializations. Such minimizers give excellent approximations to the rows of X0 . The rows are then recovered by a linear programming rounding and deflation. [ABSTRACT FROM AUTHOR]

Although there are vast potential locations to install FACTS devices in a power system, the actual installation number is very limited due to economical consideration. Therefore the allocation strategy exhibits strong sparsity. This paper formulates FACTS device allocation problem as a general sparsity-constrained OPF problem and employs Lq(0< q\leq 1) norms to enforce sparsity on FACTS devices setting values to achieve solutions with desirable device numbers and sites. An algorithm based on alternating direction method of multipliers is proposed to solve the sparsity-constrained OPF problem. The algorithm exploits the separability structure and decomposes the original problem into an NLP subproblem, an Lq regularization subproblem, and a simple dual variable update step. The NLP subproblem is solved by the interior point method. The Lq regularization subproblem has a closed-form solution expressed by shrinkage-threholding operators. The convergence of the proposed method is theoretically analyzed and discussed. The proposed method is successfully tested on allocation of SVC, TCSC, and TCPS on IEEE 30-, 118-, and 300-bus systems. Case studies are presented and discussed for both single-type and multiple-type FACTS devices allocation problems, which demonstrates the effectiveness and efficiency of the proposed formulation and algorithm. [ABSTRACT FROM PUBLISHER]

Distributed desynchronization algorithms are key to wireless sensor networks as they allow for medium access control in a decentralized manner. In this paper, we view desynchronization primitives as iterative methods that solve optimization problems. In particular, by formalizing a well established desynchronization algorithm as a gradient descent method, we establish novel upper bounds on the number of iterations required to reach convergence. Moreover, by using Nesterov's accelerated gradient method, we propose a novel desynchronization primitive that provides for faster convergence to the steady state. Importantly, we propose a novel algorithm that leads to decentralized time-synchronous multichannel TDMA coordination by formulating this task as an optimization problem. Our simulations and experiments on a densely-connected IEEE 802.15.4-based wireless sensor network demonstrate that our scheme provides for faster convergence to the steady state, robustness to hidden nodes, higher network throughput and comparable power dissipation with respect to the recently standardized IEEE 802.15.4e-2012 time-synchronized channel hopping (TSCH) scheme. [ABSTRACT FROM AUTHOR]

Iterative image reconstruction for positron emission tomography can improve image quality by using spatial regularization. The most commonly used quadratic penalty often oversmoothes sharp edges and fine features in reconstructed images, while nonquadratic penalties can preserve edges and achieve higher contrast recovery. Existing optimization algorithms such as the expectation maximization (EM) and preconditioned conjugate gradient (PCG) algorithms work well for the quadratic penalty, but are less efficient for high-curvature or nonsmooth edge-preserving regularizations. This paper proposes a new algorithm to accelerate edge-preserving image reconstruction by using two strategies: trust surrogate and optimization transfer descent. Trust surrogate approximates the original penalty by a smoother function at each iteration, but guarantees the algorithm to descend monotonically; Optimization transfer descent accelerates a conventional optimization transfer algorithm by using conjugate gradient and line search. Results of computer simulations and real 3-D data show that the proposed algorithm converges much faster than the conventional EM and PCG for smooth edge-preserving regularization and can also be more efficient than the current state-of-art algorithms for the nonsmooth \ell 1 regularization. [ABSTRACT FROM PUBLISHER]