Search

In this study, we extended the applicability of a derivative-free algorithm to encompass the solution of operators that may be either differentiable or non-differentiable. Conditions weaker than the ones in earlier studies are employed for the convergence analysis. The earlier results considered assumptions up to the existence of the ninth order derivative of the main operator, even though there are no derivatives in the algorithm, and the Taylor series on the finite Euclidian space restricts the applicability of the algorithm. Moreover, the previous results could not be used for non-differentiable equations, although the algorithm could converge. The new local result used only conditions on the divided difference in the algorithm to show the convergence. Moreover, the more challenging semi-local convergence that had not previously been studied was considered using majorizing sequences. The paper included results on the upper bounds of the error estimates and domains where there was only one solution for the equation. The methodology of this paper is applicable to other algorithms using inverses and in the setting of a Banach space. Numerical examples further validate our approach. [ABSTRACT FROM AUTHOR]

In this paper, a high-accuracy conservative implicit algorithm for computing the space fractional coupled Schrödinger–Boussinesq system is constructed. Meanwhile, the conservative nature, a priori boundedness, and solvability of the numerical solution are presented. Then, the proposed algorithm is proved to be second-order convergence in temporal and fourth-order spatial convergence using the discrete energy method. Finally, some numerical experiments validate the effectiveness of the conservative algorithm and confirm the accuracy of the theoretical results for different choices of the fractional-order α. [ABSTRACT FROM AUTHOR]

Iterative algorithms requiring the computationally expensive in general inversion of linear operators are difficult to implement. This is the reason why hybrid Newton-like algorithms without inverses are developed in this paper to solve Banach space-valued nonlinear equations. The inverses of the linear operator are exchanged by a finite sum of fixed linear operators. Two types of convergence analysis are presented for these algorithms: the semilocal and the local. The Fréchet derivative of the operator on the equation is controlled by a majorant function. The semi-local analysis also relies on majorizing sequences. The celebrated contraction mapping principle is utilized to study the convergence of the Krasnoselskij-like algorithm. The numerical experimentation demonstrates that the new algorithms are essentially as effective but less expensive to implement. Although the new approach is demonstrated for Newton-like algorithms, it can be applied to other single-step, multistep, or multipoint algorithms using inverses of linear operators along the same lines. [ABSTRACT FROM AUTHOR]

As a universal approximator, the first order Takagi–Sugeno fuzzy system possesses the capability to approximate widespread nonlinear systems through a group of IF THEN fuzzy rules. Although group lasso regularization has the advantage of inducing group sparsity and handling variable selection issues, it can lead to numerical oscillations and theoretical challenges in calculating the gradient at the origin when employed directly during training. The paper addresses the aforementioned obstacle by invoking a smoothing function to approximate group lasso regularization. On this basis, a gradient‐based neuro fuzzy learning algorithm with smoothing group lasso regularization for the first order Takagi–Sugeno fuzzy system is proposed. The convergence of the proposed algorithm is rigorously proved under gentle conditions. In addition, experimental outcomes acquired on two approximations and two classification simulations demonstrate that the proposed algorithm outperforms the algorithm with original group lasso regularization and L2 regularization in terms of error, pruned neurons, and accuracy. This is particularly evident in significant advancements in pruned neurons due to group sparsity. In comparison to the algorithm with L2 regularization, the proposed algorithm exhibits improvements of 6.3, 5.3, and 142.6 in pruned neurons during sin(πx)$(\pi x)$ function, Gabor function, and Sonar benchmark dataset simulations, respectively. [ABSTRACT FROM AUTHOR]

In this paper, we develop a Jacobi-type algorithm for the approximate diagonalization of tensors of order $ d\geq 3 $ d ≥ 3 via tensor trace maximization. For a general tensor, this is an alternating least squares algorithm and the rotation matrices are chosen in each mode one-by-one to maximize the tensor trace. On the other hand, for symmetric tensors, we discuss a structure-preserving variant of this algorithm where in each iteration the same rotation is applied in all modes. We show that both versions of the algorithm converge to the stationary points of the corresponding objective functions. [ABSTRACT FROM AUTHOR]

In this paper, we apply the Anderson acceleration technique to the existing relaxation fixed‐point iteration for solving the multilinear PageRank. In order to reduce computational cost, we further consider the periodical version of the Anderson acceleration. The convergence of the proposed algorithms is discussed. Numerical experiments on synthetic and real‐world datasets are performed to demonstrate the advantages of the proposed algorithms over the relaxation fixed‐point iteration and the extrapolated shifted fixed‐point method. In particular, we give a strategy for choosing the quasi‐optimal parameters of the associated algorithms when they are applied to solve the test problems with different sizes but the same structure. [ABSTRACT FROM AUTHOR]

We propose in this paper a new minimization algorithm based on a slightly modified version of the scalar auxialiary variable (SAV) approach coupled with a relaxation step and an adaptive strategy. It enjoys several distinct advantages over popular gradient based methods: (i) it is unconditionally energy diminishing with a modified energy which is intrinsically related to the original energy, and thus no parameter tuning is needed for stability; (ii) it allows the use of large step-sizes which can effectively accelerate the convergence rate. We also present a convergence analysis for some SAV based algorithms, which include our new algorithm without the relaxation step as a special case. We apply our new algorithm to several illustrative and benchmark problems and compare its performance with several popular gradient based methods. The numerical results indicate that the new algorithm is very robust, and its adaptive version usually converges significantly faster than those popular gradient decent based methods. [ABSTRACT FROM AUTHOR]

In this paper, a fully discrete finite element scheme is established with L 1 approximation for solving the nonlinear time-fractional wave equation with a variable coefficient. The stability of the scheme is analyzed and the optimal error estimates are derived. To improve the computational efficiency, we propose a two-grid algorithm based on the fully discrete finite element scheme. With the proposed technique, a small-scale nonlinear problem is calculated by iteration in a coarse-grid space with mesh size H , and then a linearized problem is solved in a fine-grid space with mesh size h , H ≫ h. This technique not only maintains the numerical precision, but also saves a lot of computing time. The stability and optimal convergence order are considered. Using the Taylor expansion, the second two-grid algorithm is constructed to improve the optimal convergence order. The similar properties are discussed in detail. Numerical experiments are provided to illustrate the theoretical analysis and test the performance of the proposed algorithms. [ABSTRACT FROM AUTHOR]

The GSCP (Gauss–Seidel based projection‐contraction) algorithm has been well established in solving elastoplastic mixed complementarity problems (MiCPs) or other problems which have similar mathematical form. However, the stability problems in the GSPC algorithm may cause nonconvergence or program crashes. Thus, the method of constructing and analyzing counterexamples is adopted to study the reasons for the instability of the GSPC algorithm. The R‐GSPC algorithm was developed by modifying the part of the GSPC algorithm that caused the crash to make the algorithm stable. In addition, this paper analyzes the convergence, compatibility, and initial value impact of R‐GSPC, and finds that the R‐GSPC algorithm greatly improves the robustness and stability without losing accuracy. This paper allows the GSPC algorithm to be better extended to more fields and to be suitabled for being introduced into commercial programs. [ABSTRACT FROM AUTHOR]

In this paper, we propose a fast Alikhanov algorithm with nonuniform time steps for a two dimensional distributed‐order time–space fractional advection–dispersion equation. First, an efficient fast Alikhanov algorithm on the general nonuniform time steps for the evaluation of Caputo fractional derivative is presented to sharply reduce the computational work and storage, and are applied to the distributed‐order time fractional derivative or multi‐term time fractional derivative under the nonsmooth regularity assumptions. And a generalized discrete fractional Grönwall inequality is extended to multi‐term fractional derivative or distributed‐order fractional derivative for analyzing theoretically our algorithm. Then the stability and convergence of time semi‐discrete scheme are investigated. Furthermore, we derive the corresponding fully discrete scheme by finite element method and discuss its convergence. At last, the given numerical examples adequately confirm the correctness of theoretical analysis and compare the computing effectiveness between the fast algorithm and the direct method. [ABSTRACT FROM AUTHOR]

Most genetic operators use random mating selection strategy and fixed rate crossover operator to solve various optimization problems. In order to improve the convergence and diversity of the algorithm, an adaptive adjacent maximum distance crossover operator is proposed in this paper. A new mating selection strategy (distance-based mating selection strategy) and an adaptive mechanism (adaptive crossover strategy based on population convergence) are adopted. Distance-based mating selection strategy purposefully selects parents to produce better offspring. Adaptive crossover strategy based on population convergence increases the convergence speed of the algorithm by controlling the crossover probability. The proposed crossover strategy is evaluated on the simulated binary crossover operators of non-dominated sorting genetic algorithm II and multi-objective evolutionary algorithm based on decomposition. The performance of the algorithm is verified on a series of standard test problems. Finally, the optimization results of the improved algorithm using adaptive adjacent maximum distance crossover operator and the standard algorithm are compared and analyzed. The experimental results show that the algorithm using adaptive adjacent maximum distance crossover operator has better optimization results. [ABSTRACT FROM AUTHOR]

In this paper, we prove that a three-step iterative algorithm converges strongly to the solution of a functional Volterra integral equation of the second kind. We also show the solution presented in the convergence theorem can obtain by removing a strong restriction imposed on the control sequence. Finally, we analyse the data-dependence result by using this iterative algorithm, and to support obtained result we give an example. [ABSTRACT FROM AUTHOR]

As an algorithmic framework, message passing is extremely powerful and has wide applications in the context of different disciplines including communications, coding theory, statistics, signal processing, artificial intelligence and combinatorial optimization. In this paper, we investigate the performance of a message-passing algorithm called min-sum belief propagation (BP) for the vertex-disjoint shortest $k$ -path problem ($k$ -VDSP) on weighted directed graphs, and derive the iterative message-passing update rules. As the main result of this paper, we prove that for a weighted directed graph $G$ of order $n$ , BP algorithm converges to the unique optimal solution of $k$ -VDSP on $G$ within $O(n^{2}w_{max})$ iterations, provided that the weight $w_{e}$ is nonnegative integral for each arc $e\in E(G)$ , where $w_{max}=\max \{w_{e}: e\in E(G)\}$. To the best of our knowledge, this is the first instance where BP algorithm is proved correct for NP-hard problems. Additionally, we establish the extensions of $k$ -VDSP to the case of multiple sources or sinks. [ABSTRACT FROM AUTHOR]

As a classical machine learning model, support vector machine (SVM) has attracted much attention due to its rigorous theoretical foundation and powerful discriminative performance. The doubly regularized SVM (DRSVM) is an important variant of SVM based on elastic-net regularization, which considers both the sparsity and stability of the model. To tackle the problems of explosive increases in data dimensions and data volume, the alternating direction method of multipliers (ADMM) algorithm can be used to train the DRSVM model. ADMM is an effective iterative algorithm for solving convex optimization problems by decomposing a large issue into a series of solvable subproblems, which is also well suited for distributed computing. However, lack of guaranteed convergence and slow convergence rate are two critical limitations of ADMM. In this paper, a 3-block ADMM algorithm based on the over-relaxation technique is proposed to accelerate DRSVM training, namely, the over-relaxed DRSVM (O-RDRSVM). The main strategy of the over-relaxation technique is to further append the information from the previous iteration to the next iteration to improve the convergence of ADMM. We also propose a distributed version of O-RDRSVM to handle parallel and distributed computing faster, termed DO-RDRSVM. Moreover, we develop a fast O-RDRSVM algorithm (FO-RDRSVM) and a fast DO-RDRSVM algorithm (FDO-RDRSVM), which further reduce the computational cost of O-RDRSVM and DO-RDRSVM by employing the matrix inversion lemma. The convergence analyses ensure the effectiveness of our algorithms for DRSVM training. Finally, extensive experiments on public datasets demonstrate the advantages of our algorithms in terms of convergence rate and training time while maintaining accuracy and sparsity comparable to those of previous works. [ABSTRACT FROM AUTHOR]

For a general third-order tensor A ∈ R n × n × n the paper studies two closely related problems, an SVD-like tensor decomposition and an (approximate) tensor diagonalization. We develop a Jacobi-type algorithm that works on 2 × 2 × 2 subtensors and, in each iteration, maximizes the sum of squares of its diagonal entries. We show how the rotation angles are calculated and prove convergence of the algorithm. Different initializations of the algorithm are discussed, as well as the special cases of symmetric and antisymmetric tensors. The algorithm can be generalized to work on higher-order tensors. [ABSTRACT FROM AUTHOR]

In this paper, an implicit viscosity iterative algorithm for nonexpansive mappings is proposed and investigated on Hadamard manifolds. A convergence theorem of a fixed point of a nonexpansive mapping is established on Hadamard manifolds. It is proved that the fixed point also solves a variational inequality. [ABSTRACT FROM AUTHOR]

For multilayer perceptron (MLP), the initial weights will significantly influence its performance. Based on the enhanced fractional derivative extend from convex optimization, this paper proposes a fractional gradient descent (RFGD) algorithm robust to the initial weights of MLP. We analyze the effectiveness of the RFGD algorithm. The convergence of the RFGD algorithm is also analyzed. The computational complexity of the RFGD algorithm is generally larger than that of the gradient descent (GD) algorithm but smaller than that of the Adam, Padam, AdaBelief, and AdaDiff algorithms. Numerical experiments show that the RFGD algorithm has strong robustness to the order of fractional calculus which is the only added parameter compared to the GD algorithm. More importantly, compared to the GD, Adam, Padam, AdaBelief, and AdaDiff algorithms, the experimental results show that the RFGD algorithm has the best robust performance for the initial weights of MLP. Meanwhile, the correctness of the theoretical analysis is verified. [ABSTRACT FROM AUTHOR]

In view of the problems that node clock drift， network link delay and poor synchronization ability in Ethernet， this paper constructs a clock synchronization system model based on the communication principle of RTEthernet protocol. By studying the composition and working principle of RTEthernet， three factors affecting the time synchronization precision of real-time Ethernet-drift rate， network transmission delay and clock Byzantine fault are considered. Based on this， the original FTA clock synchronization algorithm is analyzed， and it is found that its fault tolerance performance is obviously reduced when Byzantine faults increase. The idea of "median fault tolerance" is introduced to improve it and RTE-FTM algorithm is proposed. Through the CANoe simulation platform， two Byzantine faults and no Byzantine faults in the system （seven nodes） are compared and analyzed， and the precision loss rate of the system is reduced by 3.1%， which verifies the convergence and effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

This paper presents a new numerical algorithm dealing with multi‐term time‐fractional Burgers‐type equation involving the Caputo derivative. The proposed method consists of temporal discretization of L2$$ L2 $$ formula and spatial discretization using the exponential B‐splines. The semi implicit approach is applied to discretize the nonlinear term u∂xu$$ u{\partial}_{\mathtt{x}}u $$. We adopt the Von–Neumann method to study stability. We also establish the convergence analysis. The proposed method is employed to solve a few numerical examples in order to test its efficiency and accuracy. Comparisons with the recent works confirm the efficiency and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, efficient methods are derived for seeking a numerical solution to the time-fractional convection–diffusion equation whose solution very likely exhibits a weak regularity at the starting time. Here, the time-fractional derivative in the Caputo sense with order in (0 , 1) is discretized by the L 2 - 1 σ methods with uniform and non-uniform meshes and the spatial derivative is approximated by the local discontinuous Galerkin (LDG) finite element methods. The fully discrete schemes for both situations are established and analyzed. It is shown that the derived schemes are numerically stable and convergent. Finally, some numerical examples are performed to testify the effectiveness of the obtained algorithm. [ABSTRACT FROM AUTHOR]

Gossip protocols are popular methods for average consensus problems in distributed computing. We prove new convergence guarantees for a variety of such protocols, including path, clique, and synchronous pairwise gossip. These arise by exploiting the connection between these protocols and the block randomized Kaczmarz method for solving linear systems. Moreover, we extend existing convergence results for block randomized Kaczmarz to allow for a more general choice of blocks, rank-deficient systems, and provide a tighter convergence rate guarantee. Previous results for block randomized Kaczmarz assumed blocks formed a paving of the systems; our analysis, contrary to the title of our paper, generalizes to a broader class of blocks that we call coverings. We furthermore apply this analysis to inconsistent consensus models and obtain similar guarantees. An extensive empirical analysis of these methods is provided for a variety of synthetic and real networks. [ABSTRACT FROM AUTHOR]

This paper studies distributed algorithms for (strongly convex) composite optimization problems over mesh networks, subject to quantized communications. Instead of focusing on a specific algorithmic design, a black-box model is proposed, casting linearly convergent distributed algorithms in the form of fixed-point iterates. The algorithmic model is equipped with a novel random or deterministic Biased Compression (BC) rule on the quantizer design, and a new Adaptive encoding Non-uniform Quantizer (ANQ) coupled with a communication-efficient encoding scheme, which implements the BC-rule using a finite number of bits (below machine precision). This fills a gap existing in most state-of-the-art quantization schemes, such as those based on the popular compression rule, which rely on communication of some scalar signals with negligible quantization error (in practice quantized at the machine precision). A unified communication complexity analysis is developed for the black-box model, determining the average number of bits required to reach a solution of the optimization problem within a target accuracy. It is shown that the proposed BC-rule preserves linear convergence of the unquantized algorithms, and a trade-off between convergence rate and communication cost under ANQ-based quantization is characterized. Numerical results validate our theoretical findings and show that distributed algorithms equipped with the proposed ANQ have more favorable communication cost than algorithms using state-of-the-art quantization rules. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the convergence speed of the Arimoto-Blahut algorithm. For many channel matrices, the convergence speed is exponential, but for some channel matrices it is slower than exponential. By analyzing the Taylor expansion of the defining function of the Arimoto-Blahut algorithm, we will make the conditions clear for the exponential or slower convergence. The analysis of the slow convergence in this paper is new. Based on this analysis, we will compare the convergence speeds of the Arimoto-Blahut algorithm numerically with the values obtained in our theorems for several channel matrices. The purpose of this paper is to obtain a complete understanding of the convergence speed of the Arimoto-Blahut algorithm. [ABSTRACT FROM AUTHOR]

In this paper we intend to construct a structure preserving difference scheme for two-dimensional space-fractional nonlinear Schrödinger (2D SFNS) equation with the integral fractional Laplacian. The temporal direction is discretized by the modified Crank-Nicolson method, and the spatial variable is approximated by a novel fractional central difference method. The mass and energy conservations and the convergence are rigorously proved for the proposed scheme. For 1D SFNS equation, the convergence relies heavily on the L ∞ -norm boundness of the numerical solution of the proposed scheme. However, we cannot obtain the L ∞ -norm boundness of the numerical solution by using the similar process for the 2D SFNS equation. One of the major significance of this paper is that we first obtain the L ∞ -norm boundness of the numerical solution and L 2 -norm error estimate via the popular "cut-off" function for the 2D SFNS equation. Further, we reveal that the spatial discretization generates a block-Toeplitz coefficient matrix, and it will be ill-conditioned as the spatial grid mesh number M and the fractional order α increase. Thus, we exploit an linearized iteration algorithm for the nonlinear system, so that it can be efficiently solved by the Krylov subspace solver with a suitable preconditioner, where the 2D fast Fourier transform (2D FFT) is applied in the solver to accelerate the matrix-vector product, and the standard orthogonal projection approach is used to eliminate the drift of mass and energy. Extensive numerical results are reported to confirm the theoretical analysis and high efficiency of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In real-world applications, it is important for machine learning algorithms to be robust against data outliers or corruptions. In this paper, we focus on improving the robustness of a large class of learning algorithms that are formulated as low-rank semi-definite programming (SDP) problems. Traditional formulations use the square loss, which is notorious for being sensitive to outliers. We propose to replace this with more robust noise models, including the $\ell _1$ ℓ 1 -loss and other nonconvex losses. However, the resultant optimization problem becomes difficult as the objective is no longer convex or smooth. To alleviate this problem, we design an efficient algorithm based on majorization-minimization. The crux is on constructing a good optimization surrogate, and we show that this surrogate can be efficiently obtained by the alternating direction method of multipliers (ADMM). By properly monitoring ADMM's convergence, the proposed algorithm is empirically efficient and also theoretically guaranteed to converge to a critical point. Extensive experiments are performed on four machine learning applications using both synthetic and real-world data sets. Results show that the proposed algorithm is not only fast but also has better performance than the state-of-the-arts. [ABSTRACT FROM AUTHOR]

This paper investigates the stochastic optimization problem focusing on developing scalable parallel algorithms for deep learning tasks. Our solution involves a reformation of the objective function for stochastic optimization in neural network models, along with a novel parallel computing strategy, coined the weighted aggregating stochastic gradient descent (WASGD). Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers. Without any center variable, the new method automatically gauges the importance of local workers and accepts them by their contributions. Furthermore, we have developed an enhanced version of the method, WASGD+, by (1) implementing a designed sample order and (2) upgrading the weight evaluation function. To validate the new method, we benchmark our pipeline against several popular algorithms including the state-of-the-art deep neural network classifier training techniques (e.g., elastic averaging SGD). Comprehensive validation studies have been conducted on four classic datasets: CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST. Subsequent results have firmly validated the superiority of the WASGD scheme in accelerating the training of deep architecture. Better still, the enhanced version, WASGD+, is shown to be a significant improvement over its prototype. [ABSTRACT FROM AUTHOR]

In this paper, we describe iterative derivative-free algorithms for multiple roots of a nonlinear equation. Many researchers have evaluated the multiple roots of a nonlinear equation using the first- or second-order derivative of functions. However, calculating the function's derivative at each iteration is laborious. So, taking this as motivation, we develop second-order algorithms without using the derivatives. The convergence analysis is first carried out for particular values of multiple roots before coming to a general conclusion. According to the Kung–Traub hypothesis, the new algorithms will have optimal convergence since only two functions need to be evaluated at every step. The order of convergence is investigated using Taylor's series expansion. Moreover, the applicability and comparisons with existing methods are demonstrated on three real-life problems (e.g., Kepler's, Van der Waals, and continuous-stirred tank reactor problems) and three standard academic problems that contain the root clustering and complex root problems. Finally, we see from the computational outcomes that our approaches use the least amount of processing time compared with the ones already in use. This effectively displays the theoretical conclusions of this study. [ABSTRACT FROM AUTHOR]

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive modified weak Galerkin (AMWG) method is considered to solve second-order elliptic problem. Under the assumption of a penalty parameter, by showing reliability of error estimator, comparison of solutions and reduction of error estimator, the sum of the energy error and the scaled error estimator, between two consecutive adaptive loops, is proved to be a contraction, namely, the adaptive algorithm is convergent. Numerical experiments are implemented to support the theoretical results. [ABSTRACT FROM AUTHOR]

It is common for communities in real-world networks to possess hierarchical and overlapping structures, which make community detection even more challenging. In this paper, by investigating consensus process of the classical DeGroot model in opinion dynamics, we propose a novel method based on the cumulative opinion distance (COD) to discover hierarchical and overlapping communities. It is shown that this method is different from those classical algorithms relying on static fitness metrics that depict the inhomogeneous connectivity across the network. The proposed method is validated from two aspects. First, by estimating the eigenvectors of adjacency matrices, we investigate the detectability limit of our algorithms on random networks, which together with the results concerning the convergence speed of consensus guarantees the performance of our method theoretically. Second, experiments on both large scale real-world networks and artificial benchmarks show that our method is very effective and competitive on hierarchical modular graphs. In particular, it outperforms the state-of-the-art algorithms on overlapping community detection. [ABSTRACT FROM AUTHOR]

Dual Coordinate Descent (DCD) is one of the most popular optimization methods. The parallelization of DCD is difficult, as DCD is sequential in nature. As such, simultaneously running multiple DCD threads on batches of data elements causes result inaccuracy and slow convergence, due to the concurrent updates of multiple coordinates. Some parallelization methods adopt separable approximate functions that depend on the degree of parallelism. Such dependencies result in both poor scalability and slow-convergence. To address these challenges, in this paper we present a new parallel primal-dual algorithm for DCD, called PPD. In PPD, the block data distribution is utilized to obtain a new approximate function that is independent from the parallelism. Moreover, PPD is designed with a novel primal-dual acceleration scheme to approach the optimal solution closely and quickly. We demonstrate the advantages of PPD in terms of scalability and efficiency through experiments. [ABSTRACT FROM AUTHOR]

In order to improve the convergence and diversity of the non-dominated solution set of multi-objective optimization problems, and solve the problem that the algorithm is easy to fall into the local optimum in the later stage, according to the characteristics of different differential evolution strategies, this paper added random disturbances, and proposed a decomposition multi-objective evolution algorithm with adaptive differential evolution strategy based on the improved Chebyshev mechanism (MOEA/D-ADE-levy). Firstly, it used the mixed-level orthogonal experiment to generate uniform weight vectors and applied to improve the Chebyshev mechanism to decompose the sub-problems to obtain a uniformly distributed initial population. Secondly, it divided the population into excellent individuals, intermediate individuals and poor individuals, and used different individuals. The mutation strategy used an adaptive mechanism for the mutation factor F and the crossover probability CR to improve the convergence and diversity of the non-dominated solution set. Finally, it added the Lévy random perturbation to the solution set that fell into the local optimum to increase its global search ability, and jumped out of the local optimum. The DTLZ test function was used to verify the effectiveness of the algorithm. And the proposed algorithm was compared with common algorithms such as NSGA2, NSGA3, MOEA/D, MOEA/D-DE, etc.. And the diversity and convergence analysis of the algorithm was performed using GD and IGD evaluation indicators. The results show that the algorithm has been improved and improved in terms of convergence and diversity, and can obtain a better Pareto solution set. [ABSTRACT FROM AUTHOR]

We consider a class of nonconvex regularized optimization problems, which appear frequently in machine learning and data processing. Due to the structure of the problems, the iteratively reweighted algorithm was developed and applied to the consensus optimization. In this paper, we propose the acceleration of this scheme by adding an inertial term in each iteration. The proposed algorithms inherit the advantages of classical decentralized algorithms: they can be implemented over a connected network, in which the agents communicate with their neighbors and perform local computations. We also employ the diminishing stepsizes technique for the iteratively reweighted algorithm and consider its acceleration. In specific cases, our algorithms reduce to existing decentralized schemes and also indicate novel ones. Mathematically, we prove the convergence for both algorithms with several assumptions on the objective functions. With Kurdyka-Łojasiewicz property, convergence rates can be derived for constant stepsize case. Numerical results demonstrate the efficiency of the algorithms. [ABSTRACT FROM AUTHOR]

Box-constrained optimization problems in the real m × n matrix space have been widely applied in big data mining. However, efficient solution of them is still a challenge. In this paper, a new nonmonotone line search rule is first proposed by extending the well-known ones and inheriting their advantages. Then, by analyzing and exploiting properties of this rule, a new nonmonotone spectral projected gradient algorithm is developed to solve the box-constrained optimization problems in the matrix space. Global convergence of the developed algorithm is also established. Numerical tests are conducted on a series of randomly generated test problems and those in the set of benchmark test problems. Compared with other existing nonmonotone line search rules, our rule shows its advantages in terms of the significantly reduced number of function evaluations and significantly reduced number of iterations. To further validate applicability of this research, we apply the studied optimization problem and the developed algorithm to solve the problems of image clustering. Numerical results demonstrate that the proposed method can generate better clustering results and is more robust than the similar ones available in the literature. [ABSTRACT FROM AUTHOR]

Online gradient descent method has been widely applied for parameter learning in neuro-fuzzy systems. The success of the application relies on the convergence of the learning procedure. However, there barely have been convergence analyses on the online learning procedure for neuro-fuzzy systems. In this paper, an online gradient learning algorithm with adaptive learning rate is proposed to identify the parameters of the neuro-fuzzy systems representing the Mamdani fuzzy model with Gaussian fuzzy sets. We take the reciprocals of the variances of the Gaussian membership functions, rather than the variances themselves, as independent variables when computing the gradient with respect to the variance parameters. Subsequently, oscillation of the gradient value in the learning process can be avoided. Furthermore, some convergence results for this online learning scheme are studied. Finally, three numerical examples are provided to illustrate the performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we propose to extend the standard Convolutional Dictionary Learning problem to a tensor representation where the activations are constrained to be “low-rank” through a Canonical Polyadic decomposition. We show that this additional constraint increases the robustness of the CDL with respect to noise and improve the interpretability of the final results. In addition, we discuss in detail the advantages of this representation and introduce two algorithms, based on ADMM or FISTA, that efficiently solve this problem. We show that by exploiting the low rank property of activations, they achieve lower complexity than the main CDL algorithms. Finally, we evaluate our approach on a wide range of experiments, highlighting the modularity and the advantages of this tensorial low-rank formulation. [ABSTRACT FROM AUTHOR]

The use of a large-scale antenna array (LSAA) has become an important characteristic of multi-antenna communication systems to achieve beamforming gains such as in designing millimeter-wave (mmWave) systems to combat severe propagation losses. In such applications, each antenna element has to be driven by a radio frequency (RF) chain for the implementation of fully-digital beamformers, significantly increasing the hardware cost, complexity, and power consumption. Therefore, constant-modulus analog beamforming (CMAB) becomes a viable solution. In this paper, we consider the scaled analog beamforming (SAB) or constant-modulus analog beamforming (CMAB) architecture and design the system parameters by solving two variants of beampattern matching problem. In the first case, both the magnitude and phase of the beampattern are matched to the given desired beampattern whereas in the second case, only the magnitude of the beampattern is matched. Both the beampattern matching problems are cast as a variant of the constant-modulus least-squares (CLS) problem. We provide efficient algorithms based on the alternating majorization-minimization (AMM) framework that combines the alternating minimization and the MM frameworks and the conventional-cyclic coordinate descent (C-CCD) algorithms to solve the problem in each case. We also propose algorithms based on a new modified-CCD (M-CCD) based approach. For all the developed algorithms we prove convergence to a Karush-Kuhn-Tucker (KKT) point (or a stationary point). Numerical results demonstrate that the proposed algorithms converge faster than the state-of-the-art solutions. Among all the algorithms, the M-CCD-based algorithms have faster convergence when evaluated in terms of the number of iterations and the AMM-based algorithms offer lower complexity. [ABSTRACT FROM AUTHOR]

The multiple-sets split feasibility problem is the generalization of split feasibility problem, which has been widely used in fuzzy image reconstruction and sparse signal processing systems. In this paper, we present an inertial relaxed algorithm to solve the multiple-sets split feasibility problem by using an alternating inertial step. The advantage of this algorithm is that the choice of stepsize is determined by Armijo-type line search, which avoids calculating the norms of operators. The weak convergence of the sequence obtained by our algorithm is proved under mild conditions. In addition, the numerical experiments are given to verify the convergence and validity of the algorithm. [ABSTRACT FROM AUTHOR]

• The fractional Cattaneo model is formulated to describe the heat flow in a porous medium. • The hybrid scheme based on the RBF-PU method is proposed to approximate the model. • Unconditional stability and convergence of the time discretization formulation are proved using energy method. • Numerical results are conducted to validate the theoretical findings. The generalized Cattaneo model describes the heat conduction system in the perspective of time-nonlocality. This paper proposes an accurate and robust meshless technique for approximating the solution of the time fractional Cattaneo model applied to the heat flow in a porous medium. The fractional derivative is formulated in the Caputo sense with order 1 < α < 2. First, a finite difference technique of convergence order O (δ t 3 − α ) is adopted to achieve the temporal discretization. The unconditional stability of the method and its convergence are analysed using the discrete energy technique. Then, a local meshless method based on the radial basis function partition of unity collocation is employed to obtain a full discrete algorithm. The matrices produced using this localized scheme are sparse and, therefore, they are not subject to ill-conditioning and do not pose a large computational burden. Two examples illustrate in computational terms of the accuracy and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper is concerned with numerical solutions of one-dimensional (1D) and two-dimensional (2D) nonlinear coupled Schrödinger-Boussinesq equations (CSBEs) by a type of linearly energy- and mass- preserving finite difference methods (EMP-FDMs) because the existing EMP-FDMs for CSBEs are nonlinear and time-consuming, and corresponding theoretical analyses are not easy to generalize high-dimensional problems. Firstly, a linearized EMP-FDM is created for solving 1D CSBEs. By using the discrete energy analysis method, it is shown that this scheme is uniquely solvable and convergent with an order of O (Δ t 2 + h x 2) in L ∞ -, H 1 - and L 2 -norms, and corresponding numerical energy and mass are conservative. Then, by generalizing this type of EMP-FDM, a linearized EMP-FDM is developed for solving 2D CSBEs. Theoretical findings including the convergence, the discrete conservative laws, and the solvability of this numerical algorithm are strictly derived in detail by using the discrete energy analysis method as well. Finally, numerical results confirm the efficiency of our algorithms and the exactness of the theoretical results. [ABSTRACT FROM AUTHOR]

While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approach, this paper presents a new (stochastic) distributed algorithm with Nesterov momentum for accelerated optimization of non-convex and non-smooth problems. Theoretically, we show that the proposed algorithm can achieve an $\epsilon$ -stationary solution under a constant step size with $\mathcal {O}(1/\epsilon ^2)$ computation complexity and $\mathcal {O}(1/\epsilon)$ communication complexity when the epigraph of the non-smooth term is a polyhedral set. When compared to the existing gradient tracking based methods, the proposed algorithm has the same order of computation complexity but lower order of communication complexity. To the best of our knowledge, the presented result is the first stochastic algorithm with the $\mathcal {O}(1/\epsilon)$ communication complexity for non-convex and non-smooth problems. Numerical experiments for a distributed non-convex regression problem and a deep neural network based classification problem are presented to illustrate the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

This paper studies the convergence properties of the well-known message-passing algorithm for convex optimisation. Under the assumption of pairwise separability and scaled diagonal dominance, asymptotic convergence is established and a simple bound for the convergence rate is provided for message-passing. In comparison with previous results, our results do not require the given convex program to have known convex pairwise components and that our bound for the convergence rate is tighter and simpler. When specialised to quadratic optimisation, we generalise known results by providing a very simple bound for the convergence rate. [ABSTRACT FROM AUTHOR]

In this paper, we consider the policy evaluation problem in reinforcement learning with agents on a decentralized and directed network. In order to evaluate the quality of a fixed policy in this decentralized setting, one option is for agents to run decentralized temporal-difference (TD) collaboratively. To account for the practical scenarios where the state and action spaces are large and malicious attacks emerge, we focus on the decentralized TD learning with linear function approximation in the presence of malicious agents (often termed as Byzantine agents). We propose a trimmed mean-based Byzantine-resilient decentralized TD algorithm to perform policy evaluation in this setting. We establish the finite-time convergence rate, as well as the asymptotic learning error that depends on the number of Byzantine agents. Numerical experiments corroborate the robustness of the proposed algorithm. [ABSTRACT FROM AUTHOR]