Search

The optimization problems with a sparsity constraint is a class of important global optimization problems. A typical type of thresholding algorithms for solving such a problem adopts the traditional full steepest descent direction or Newton-like direction as a search direction to generate an iterate on which a certain thresholding is performed. Traditional hard thresholding discards a large part of a vector, and thus some important information contained in a dense vector has been lost in such a thresholding process. Recent study (Zhao in SIAM J Optim 30(1): 31–55, 2020) shows that the hard thresholding should be applied to a compressible vector instead of a dense vector to avoid a big loss of information. On the other hand, the optimal k-thresholding as a novel thresholding technique may overcome the intrinsic drawback of hard thresholding, and performs thresholding and objective function minimization simultaneously. This motivates us to propose the so-called partial gradient optimal thresholding (PGOT) method and its relaxed versions in this paper. The PGOT is an integration of the partial gradient and the optimal k-thresholding technique. The solution error bound and convergence for the proposed algorithms have been established in this paper under suitable conditions. Application of our results to the sparse optimization problems arising from signal recovery is also discussed. Experiment results from synthetic data indicate that the proposed algorithm is efficient and comparable to several existing algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we propose to extend the standard Convolutional Dictionary Learning problem to a tensor representation where the activations are constrained to be “low-rank” through a Canonical Polyadic decomposition. We show that this additional constraint increases the robustness of the CDL with respect to noise and improve the interpretability of the final results. In addition, we discuss in detail the advantages of this representation and introduce two algorithms, based on ADMM or FISTA, that efficiently solve this problem. We show that by exploiting the low rank property of activations, they achieve lower complexity than the main CDL algorithms. Finally, we evaluate our approach on a wide range of experiments, highlighting the modularity and the advantages of this tensorial low-rank formulation. [ABSTRACT FROM AUTHOR]

With growth in the number of smart devices and advancements in their hardware, in recent years, data-driven machine learning techniques have drawn significant attention. However, due to privacy and communication issues, it is not possible to collect this data at a centralized location. Federated learning is a machine learning setting where the centralized location trains a learning model over remote devices. Federated learning algorithms cannot be employed in the real world scenarios unless they consider unreliable and resource-constrained nature of the wireless medium. In this paper, we propose a federated learning algorithm that is suitable for cellular wireless networks. We prove its convergence, and provide a sub-optimal scheduling policy that improves the convergence rate. We also study the effect of local computation steps and communication steps on the convergence of the proposed algorithm. We prove, in practice, federated learning algorithms may solve a different problem than the one that they have been employed for if the unreliability of wireless channels is neglected. Finally, through numerous experiments on real and synthetic datasets, we demonstrate the convergence of our proposed algorithm. [ABSTRACT FROM AUTHOR]

In a recent paper [14], the fixed point and resolvent techniques have been used to give an iterative algorithm for solving generalized mixed equilibrium problem (GMEP). In this paper, an extended iterative algorithm, called generalized mean proximal algorithm (GMPA) is presented and its convergence to the solution of GMEP with Θ-pseudomonotne and δ- Lipschitz continuous mappings in Hilbert spaces is proved. Our approach is based on the basic notions of generalized resolvents and fixed points. The results improve and extend important recent results. [ABSTRACT FROM AUTHOR]

The dislocation hyperbolic augmented Lagrangian algorithm (DHALA) is a new approach to the hyperbolic augmented Lagrangian algorithm (HALA). DHALA is designed to solve convex nonlinear programming problems. We guarantee that the sequence generated by DHALA converges towards a Karush-Kuhn-Tucker point. We are going to observe that DHALA has a slight computational advantage in solving the problems over HALA. Finally, we will computationally illustrate our theoretical results. [ABSTRACT FROM AUTHOR]

In this paper we prove convergence of the method of successive approximations used to approximate the solution of nonlinear two-dimensional Volterra-Fredholm integral equations and define the notion of numerical stability of the algorithm with respect to the choice of the first iteration. Also we present an iterative procedure to solve such equations. Finally, the method is illustrated by solving some examples. [ABSTRACT FROM AUTHOR]

In this paper, we consider a Stokes problem arising in fluid dynamics and thermal science, which can be transformed to a symmetric saddle point problem by using the mixed finite element approximation. A relaxed non-linear inexact Uzawa algorithm is proposed for solving the problem, and the convergence of this algorithm is also considered. Numerical experiments are presented to show the efficiency of relaxed non-linear inexact Uzawa algorithm. [ABSTRACT FROM AUTHOR]

Fractional differential equations (FDEs) of distributed-order are important in depicting the models where the order of differentiation distributes over a certain range. Numerically solving this kind of FDEs requires not only discretizations of the temporal and spatial derivatives, but also approximation of the distributed-order integral, which brings much more difficulty. In this paper, based on the mid-point quadrature rule and composite two-point Gauss-Legendre quadrature rule, two finite difference schemes are established. Different from the previous works, which concerned only one- or two-dimensional problems with linear source terms, time-fractional wave equations of distributed-order whose source term is nonlinear in two and even three dimensions are considered. In addition, to improve the computational efficiency, the technique of alternating direction implicit (ADI) decomposition is also adopted. The unique solvability of the difference scheme is discussed, and the unconditional stability and convergence are analyzed. Finally, numerical experiments are carried out to verify the effectiveness and accuracy of the algorithms for both the two- and three-dimensional cases. [ABSTRACT FROM AUTHOR]

Engineering data are often highly nonlinear and contain high-frequency noise, so the Levenberg–Marquardt (LM) algorithm may not converge when a neural network optimized by the algorithm is trained with engineering data. In this work, we analyzed the reasons for the LM neural network's poor convergence commonly associated with the LM algorithm. Specifically, the effects of different activation functions such as Sigmoid, Tanh, Rectified Linear Unit (RELU) and Parametric Rectified Linear Unit (PRLU) were evaluated on the general performance of LM neural networks, and special values of LM neural network parameters were found that could make the LM algorithm converge poorly. We proposed an adaptive LM (AdaLM) algorithm to solve the problem of the LM algorithm. The algorithm coordinates the descent direction and the descent step by the iteration number, which can prevent falling into the local minimum value and avoid the influence of the parameter state of LM neural networks. We compared the AdaLM algorithm with the traditional LM algorithm and its variants in terms of accuracy and speed in the context of testing common datasets and aero-engine data, and the results verified the effectiveness of the AdaLM algorithm. [ABSTRACT FROM AUTHOR]

Estimation of the regularization parameter, which strikes a balance between the data fidelity and regularity, is essential for successfully solving ill-posed image restoration problems. Based on the classical total variation (TV) model and prevalent alternating direction method of multipliers, we hammer out a fast algorithm being able to simultaneously estimate the regularization parameter and restore the degraded image. By applying variable splitting technique to both the regularization term and data fidelity term, we overcome the nondifferentiability of TV and achieve a closed form to update the regularization parameter in each iteration. The solution is guaranteed to satisfy Morozov’s discrepancy principle. Furthermore, we present a convergence proof for the proposed algorithm on the premise of a variable regularization parameter. Experimental results demonstrate that the proposed algorithm is superior in speed and competitive in accuracy compared with several state-of-the-art methods. Besides, the proposed method can be smoothly extended to the multichannel image restoration. [ABSTRACT FROM PUBLISHER]