Search

The controlled autoregressive autoregressive moving average (CARARMA) models are of popularity to describe the evolution characteristics of dynamical systems. To overcome the identification obstacle resulting from colored noises, this paper studies the identification of the CARARMA models by forming an intermediate correlated noise model. In order to realize the real-time prediction function of the models, the on-line identification scheme is developed by constructing the dynamical objective functions based on the real-time sampled observations. Firstly, a rolling optimization cost function is built based on the observation at a single sampling instant to catch the modal information at a single time point and a generalized extended stochastic gradient (GESG) algorithm is proposed through the stochastic gradient optimization. Secondly, a rolling window cost function is built in accordance with the dynamical batch observations within data window by extending the proposed GESG algorithm and the multi-innovation generalized extended stochastic gradient algorithm is derived. Thirdly, from the perspective of theoretical analysis, the convergence proof of the proposed algorithm is provided based on the stochastic martingale convergence theory. Finally, the simulation analysis and comparison studies are provided to show the performance of the proposed algorithms. [ABSTRACT FROM AUTHOR]

The vehicle routing problem with time windows (VRPTW) is a classical optimization problem. There have been many related studies in recent years. At present, many studies have generally analyzed this problem on the two-dimensional plane, and few studies have explored it on spherical surfaces. In order to carry out research related to the distribution of goods by unmanned vehicles and unmanned aerial vehicles, this study carries out research based on the situation of a three-dimensional sphere and proposes a three-dimensional spherical VRPTW model. All of the customer nodes in this problem were mapped to the three-dimensional sphere. The chimp optimization algorithm is an excellent intelligent optimization algorithm proposed recently, which has been successfully applied to solve various practical problems and has achieved good results. The chimp optimization algorithm (ChOA) is characterized by its excellent ability to balance exploration and exploitation in the optimization process so that the algorithm can search the solution space adaptively, which is closely related to its outstanding adaptive factors. However, the performance of the chimp optimization algorithm in solving discrete optimization problems still needs to be improved. Firstly, the convergence speed of the algorithm is fast at first, but it becomes slower and slower as the number of iterations increases. Therefore, this paper introduces the multiple-population strategy, genetic operators, and local search methods into the algorithm to improve its overall exploration ability and convergence speed so that the algorithm can quickly find solutions with higher accuracy. Secondly, the algorithm is not suitable for discrete problems. In conclusion, this paper proposes an improved chimp optimization algorithm (MG-ChOA) and applies it to solve the spherical VRPTW model. Finally, this paper analyzes the performance of this algorithm in a multi-dimensional way by comparing it with many excellent algorithms available at present. The experimental result shows that the proposed algorithm is effective and superior in solving the discrete problem of spherical VRPTW, and its performance is superior to that of other algorithms. [ABSTRACT FROM AUTHOR]

In this note, a small gap is corrected in the assumption of main theorem of T.M. Tuyen (Theorem 3.1, Regularization proximal point algorithm for common fixed points of nonexpansive mappings in Banach spaces, J. Optim. Theory Appl., 152:351-365, ). We give another assumption, which allows us to obtain the strong convergence of regularization algorithms. [ABSTRACT FROM AUTHOR]

Stochastic alternating algorithms for bi-objective optimization are considered when optimizing two conflicting functions for which optimization steps have to be applied separately for each function. Such algorithms consist of applying a certain number of steps of gradient or subgradient descent on each single objective at each iteration. In this paper, we show that stochastic alternating algorithms achieve a sublinear convergence rate of O (1 / T) , under strong convexity, for the determination of a minimizer of a weighted-sum of the two functions, parameterized by the number of steps applied on each of them. An extension to the convex case is presented for which the rate weakens to O (1 / T) . These rates are valid also in the non-smooth case. Importantly, by varying the proportion of steps applied to each function, one can determine an approximation to the Pareto front. [ABSTRACT FROM AUTHOR]

Real life problems are entrenched in ambiguities. To deal with these ambiguities, stochastic functional analysis has emerged as one of the mathematical tools for solving these kinds of problems. The purpose of this paper is to extend the convergence results of deterministic convex feasibility problems to a stochastic convex feasibility problem and prove that the solution of a convex feasibility problem generated by random Mann-type and Simultaneous projection iterative algorithms with firmly non-expansive mapping converge in quadratic mean and consequently in probability to random fixed point in Hilbert space. These results extend, unify, and generalize different established deterministic results in the literature. [ABSTRACT FROM AUTHOR]

This paper is concerned with convergence of stochastic gradient algorithms with momentum terms in the nonconvex setting. A class of stochastic momentum methods, including stochastic gradient descent, heavy ball and Nesterov's accelerated gradient, is analyzed in a general framework under mild assumptions. Based on the convergence result of expected gradients, the authors prove the almost sure convergence by a detailed discussion of the effects of momentum and the number of upcrossings. It is worth noting that there are not additional restrictions imposed on the objective function and stepsize. Another improvement over previous results is that the existing Lipschitz condition of the gradient is relaxed into the condition of Hölder continuity. As a byproduct, the authors apply a localization procedure to extend the results to stochastic stepsizes. [ABSTRACT FROM AUTHOR]

The aim of this paper is to develop a method for proving almost sure convergence in the Gromov-Hausdorff-Prokhorov topology for a class of models of growing random graphs that generalises Rémy's algorithm for binary trees. We describe the obtained limits using some iterative gluing construction that generalises the famous line-breaking construction of Aldous' Brownian tree, and we characterize some of them using the self-similarity property in law that they satisfy. To do that, we develop a framework in which a metric space is constructed by gluing smaller metric spaces, called blocks, along the structure of a (possibly infinite) discrete tree. Our growing random graphs seen as metric spaces can be understood in this framework, that is, as evolving blocks glued along a growing discrete tree structure. Their scaling limit convergence can then be obtained by separately proving the almost sure convergence of every block and verifying some relative compactness property for the whole structure. For the particular models that we study, the discrete tree structure behind the construction has the distribution of an affine preferential attachment tree or a weighted recursive tree. We strongly rely on results concerning those two models and their connection, obtained in the companion paper Sénizergues (2021). [ABSTRACT FROM AUTHOR]

Abstract: In abstract of the paper [A. Rafiq, A note on “A family of methods for solving nonlinear equations”, Appl. Math. Comput. 195 (2008) 819–821] we can find the following sentences. We cite: Ujević et al. introduced a family of methods for solving nonlinear equations. However the main Algorithm 1 put forward by Ujević et al. (p. 7) is wrong. This is the main aim of this note. We also point out some major bugs in the results of Ujević et al. – the end of the citation. Here it is shown that all of the mentioned assertions are not true. In other words, the Algorithm 1 is correct (up to an obvious misprint, which is not mentioned in the above paper) and there are no major bugs in the paper by Ujević et al. In fact, these observations, which will be given in this note, show that the main aim of the paper by Rafiq is wrong. [Copyright &y& Elsevier]

The traditional matrix decomposition model cannot fully explored the intrinsic relationship between the user and the object in the mean, bias and characteristics. This paper proposed a fitting matrix model to improve the prediction performance by constructing the user and the item matrix to represent the characteristics of the user and the item respectively. The matrix decomposition model had the advantage of accuracy in the field of recommender system, but the gradient descent method, which was the most popular method to train parameters of model, had a slow convergence speed. To resolve the above defects, this paper considered to accelerate the convergence speed using the convergence of quasi Newton method, and named the proposed algorithm as fitting matrix and two orders fusion iterative (FAST) algorithm. The experimental results show that the FAST algorithm is better than the traditional non negative matrix decomposition (NMF), singular value matrix decomposition (SVD), and the regularized singular value matrix decomposition (RSVD). FAST algorithm has a decrease with regard to the mean absolute error (MAE) and the root mean square error (RMSE), and has a significant improvement in the iterative efficiency, which alleviates the problem that the accuracy is difficult to balance with the efficiency of the iteration. [ABSTRACT FROM AUTHOR]

Stochastic gradient descent algorithm is a classical and useful method for stochastic optimisation. While stochastic gradient descent has been theoretically investigated for decades and successfully applied in machine learning such as training of deep neural networks, it essentially relies on obtaining the unbiased estimates of gradients/subgradients of the objective functions. In this paper, by constructing the randomised differences of the objective function, a gradient‐free algorithm, named the stochastic randomised‐difference descent algorithm, is proposed for stochastic convex optimisation. Under the strongly convex assumption of the objective function, it is proved that the estimates generated from stochastic randomised‐difference descent converge to the optimal value with probability one, and the convergence rates of both the mean square error of estimates and the regret functions are established. Finally, some numerical examples are prsented. [ABSTRACT FROM AUTHOR]

In this paper, aiming at the unconstrained optimization problem, a new nonmonotone adaptive retrospective trust region line search method is presented, which takes advantages of multidimensional filter technique to increase the acceptance probability of the trial step. The new nonmonotone trust region ratio is presented, which based on the convex combination of nonmonotone trust region ratio and retrospective ratio. The global convergence and the superlinear convergence of the algorithm are shown in the right circumstances. Comparative numerical experiments show the better effective and robustness. [ABSTRACT FROM AUTHOR]

The stochastic Frank–Wolfe method has recently attracted much general interest in the context of optimization for statistical and machine learning due to its ability to work with a more general feasible region. However, there has been a complexity gap in the dependence on the optimality tolerance ε in the guaranteed convergence rate for stochastic Frank–Wolfe compared to its deterministic counterpart. In this work, we present a new generalized stochastic Frank–Wolfe method which closes this gap for the class of structured optimization problems encountered in statistical and machine learning characterized by empirical loss minimization with a certain type of "linear prediction" property (formally defined in the paper), which is typically present in loss minimization problems in practice. Our method also introduces the notion of a "substitute gradient" that is a not-necessarily-unbiased estimate of the gradient. We show that our new method is equivalent to a particular randomized coordinate mirror descent algorithm applied to the dual problem, which in turn provides a new interpretation of randomized dual coordinate descent in the primal space. Also, in the special case of a strongly convex regularizer our generalized stochastic Frank–Wolfe method (as well as the randomized dual coordinate descent method) exhibits linear convergence. Furthermore, we present computational experiments that indicate that our method outperforms other stochastic Frank–Wolfe methods for a sufficiently small optimality tolerance, consistent with the theory developed herein. [ABSTRACT FROM AUTHOR]

We propose in this paper a unifying scheme for several algorithms from the literature dedicated to the solving of monotone inclusion problems involving compositions with linear continuous operators in infinite dimensional Hilbert spaces. We show that a number of primal-dual algorithms for monotone inclusions and also the classical ADMM numerical scheme for convex optimization problems, along with some of its variants, can be embedded in this unifying scheme. While in the first part of the paper, convergence results for the iterates are reported, the second part is devoted to the derivation of convergence rates obtained by combining variable metric techniques with strategies based on suitable choice of dynamical step sizes. The numerical performances, which can be obtained for different dynamical step size strategies, are compared in the context of solving an image denoising problem. [ABSTRACT FROM AUTHOR]

In this paper, two modified spectral conjugate gradient methods which satisfy sufficient descent property are developed for unconstrained optimization problems. For uniformly convex problems, the first modified spectral type of conjugate gradient algorithm is proposed under the Wolfe line search rule. Moreover, the search direction of the modified spectral conjugate gradient method is sufficiently descent for uniformly convex functions. Furthermore, according to the Dai-Liao's conjugate condition, the second spectral type of conjugate gradient algorithm can generate some sufficient decent direction at each iteration for general functions. Therefore, the second method could be considered as a modification version of the Dai-Liao's algorithm. Under the suitable conditions, the proposed algorithms are globally convergent for uniformly convex functions and general functions. The numerical results show that the approaches presented in this paper are feasible and efficient. [ABSTRACT FROM AUTHOR]

Blind Source Separation(BSS) is one of the research hotspots in the field of signal processing. In order to improve the accuracy of speech recognition in driving environment, the driver's speech signal must be enhanced to improve its signal to noise ratio(SNR). Independent component analysis (ICA) algorithm is the most classical and efficient blind statistical signal processing technique. Compared with other improved ICA algorithms, fixed-point algorithm (FastICA) is well known for its fast convergence speed and good robustness. However, the convergence of FastICA algorithm is comparatively susceptible to the initial value selection of the original demixing matrix and the calculation of the iterative process is relatively large. In this paper, the gradient descent method is used to reduce the effect of initial value. What's more, the improved secant method is proposed to speed up the convergence rate and reduce the amount of computation. As the results of mixed speech separation experiment turn out, the improved algorithm is of better performance relative to the standard FastICA algorithm. Experimental results show that the proposed algorithm improves the speech quality of the target driver. It is suitable for speech separation in driving environment with low SNR. [ABSTRACT FROM AUTHOR]

A probabilistic approach to compute the geometric convergence rate of a stochastic process is introduced in this paper. The goal is to quantitatively compute both the upper and lower bounds for rate of the exponential convergence to the stationary distribution of a stochastic dynamical system. By applying the coupling method, we derive an algorithm which does not rely on the discretization of the infinitesimal generator. In this way, our approach works well for many high-dimensional examples. We apply this algorithm to the random perturbations of both iterative maps and differential equations. We show that the rate of geometric ergodicity of a random perturbed system can, to some extent, reveal the degree of chaoticity of the underlying deterministic dynamics. Various SDE models including the ones with degenerate noise or living on the high-dimensional state space are also explored. [ABSTRACT FROM AUTHOR]

Neural network learning is usually time-consuming since backpropagation needs to compute full gradients and backpropagate them across multiple layers. Despite its success of existing works in accelerating propagation through sparseness, the relevant theoretical characteristics remain under-researched and empirical studies found that they suffer from the loss of information contained in unpropagated gradients. To tackle these problems, this paper presents a unified sparse backpropagation framework and provides a detailed analysis of its theoretical characteristics. Analysis reveals that when applied to a multilayer perceptron, our framework essentially performs gradient descent using an estimated gradient similar enough to the true gradient, resulting in convergence in probability under certain conditions. Furthermore, a simple yet effective algorithm named m emorized s parse b ack p ropagation (MSBP) is proposed to remedy the problem of information loss by storing unpropagated gradients in memory for learning in the next steps. Experimental results demonstrate that the proposed MSBP is effective to alleviate the information loss in traditional sparse backpropagation while achieving comparable acceleration. [ABSTRACT FROM AUTHOR]

The Alternating Direction Multipliers Method (ADMM) is a very popular algorithm for computing the solution of convex constrained minimization problems. Such problems are important from the application point of view, since they occur in many fields of science and engineering. ADMM is a powerful numerical tool, but unfortunately its main drawback is that it can exhibit slow convergence. Several approaches for its acceleration have been proposed in the literature and in this paper we present a new general framework devoted to this aim. In particular, we describe an algorithmic framework that makes possible the application of any acceleration step while still having the guarantee of convergence. This result is achieved thanks to a guard condition that ensures the monotonic decrease of the combined residual. The proposed strategy is applied to image deblurring problems. Several acceleration techniques are compared; to the best of our knowledge, some of them are investigated for the first time in connection with ADMM. Numerical results show that the proposed framework leads to a faster convergence with respect to other acceleration strategies recently introduced for ADMM. [ABSTRACT FROM AUTHOR]

In recent years, Differential Evolution (DE) has shown excellent performance in solving optimization problems over continuous space and has been widely used in many fields of science and engineering. How to avoid the local optimal solution and how to improve the convergence performance of DE are hotpot problems for many researchers. In this paper, an improved differential evolution algorithm based on dual-strategy (DSIDE) is proposed. The DSIDE algorithm has two strategies. (1) An enhanced mutation strategy based on "DE/rand/1," which takes into account the influence of reference individuals on mutation and has strong global exploration and convergence ability. (2) A novel adaptive strategy for scaling factor and crossover probability based on fitness value has a positive impact on population diversity. The DSIDE algorithm is verified with other seven state-of-the-art DE variants under 30 benchmark functions. Furthermore, Wilcoxon sign rank-sum test, Friedman test, and Kruskal–Wallis test are utilized to analyze the results. The experiment results show that the proposed DSIDE algorithm can significantly improve the global optimization performance. [ABSTRACT FROM AUTHOR]

The long-term operation planning problem is generally solved by a chain of computational models based on Stochastic Dual Dynamic Programming (SDDP) considering a period of 5–10 years ahead. A recent issue that has arisen concerning this problem is to define a convergence criterion when using conditional value-at-risk (CVaR) with SDDP due to the lack of good upper bound estimators. The main contribution of this paper is to propose a convergence criterion in which including CVaR does not hinder convergence analysis. Also, the proposed method does not increase the computational burden. Moreover, this paper provides a method that allows forward paths to be individually assessed, which can then either be discarded to reduce computational time or even replaced in an alternative resampling scheme in the SDDP. Based on aggregate reservoir representation, the proposed method of convergence was applied on long-term operation planning problems related to the Brazilian Power System. Results showed improvements in both the SDDP technique and the effectiveness of the proposed convergence criterion when CVaR was used. [ABSTRACT FROM AUTHOR]

Summary: This paper presents some recent advances for parallel‐in‐time methods applied to linear elasticity. With recent computer architecture changes leading to stagnant clock speeds, but ever increasing numbers of cores, future speedups will be available through increased concurrency. Thus, sequential algorithms, such as time stepping, will suffer a bottleneck. This paper explores multigrid reduction in time (MGRIT) for an important application area, linear elasticity. Previously, efforts at parallel‐in‐time for elasticity have experienced difficulties, for example, the beating phenomenon. As a result, practical parallel‐in‐time algorithms for this application area currently do not exist. This paper proposes some solutions made possible by MGRIT (e.g., slow temporal coarsening and FCF‐relaxation) and, more importantly, a different formulation of the problem that is more amenable to parallel‐in‐time methods. Using a recently developed convergence theory for MGRIT and Parareal, we show that the changed formulation of the problem avoids the instability issues and allows the reduction of the error using two temporal grids. We then extend our approach to the multilevel case, where we demonstrate how slow temporal coarsening improves convergence. The paper ends with supporting numerical results showing a practical algorithm enjoying speedup benefits over the sequential algorithm. [ABSTRACT FROM AUTHOR]

The paper introduces and analyzes the convergence of a new iterative algorithm for approximating solutions of equilibrium problems involving strongly pseudomonotone and Lipschitz-type bifunctions in Hilbert spaces. The algorithm uses a stepsize sequence which is non-increasing, diminishing, and non-summable. This leads to the main advantage of the algorithm, namely that the construction of solution approximations and the proof of its convergence are done without the prior knowledge of the modulus of strong pseudomonotonicity and Lipschitz-type constants of bifunctions. The strongly convergent theorem is established under suitable assumptions. The paper also discusses the assumptions used in the formulation of the convergent theorem. Several numerical results are reported to illustrate the behavior of the algorithm with different sequences of stepsizes and also to compare it with others. [ABSTRACT FROM AUTHOR]

The paper proposes a new extragradient algorithm for solving strongly pseudomonotone equilibrium problems which satisfy a Lipschitz-type condition recently introduced by Mastroeni in auxiliary problem principle. The main novelty of the paper is that the algorithm generates the strongly convergent sequences in Hilbert spaces without the prior knowledge of Lipschitz-type constants and any hybrid method. Several numerical experiments on a test problem are also presented to illustrate the convergence of the algorithm. [ABSTRACT FROM AUTHOR]

This paper constructs a dynamic multiobjective location model; three objectives are considered: the first objective maximizes the total utility of relief supplies, the second objective minimizes the number of temporary facilities needed to operate, and the third objective maximizes the satisfaction for all demand points. We propose an improved NSGA-II to solve the optimization problem. The computational experiments are divided into two sections: In the first procedure, the numerical experiment is constructed by the classical functions ZDT1, ZDT2, and DTLZ2; the results show that the proposed algorithm generates the exact Pareto front, and the convergence and uniformity of the proposed algorithm are better than the NSGA-II and MOEA/D. In the second procedure, the simulation experiment is constructed by a case in emergency management; the results show that the proposed algorithm is more reasonable than the traditional algorithms NSGA-II and MOEA/D in terms of the three objectives. It is proved that the improved NSGA-II algorithm, which is proposed in this paper, has high precision application for the sudden disaster crisis and emergency management. [ABSTRACT FROM AUTHOR]

In this paper, we introduce an explicit parallel algorithm for finding common solutions to a system of variational inequalities over the set of common fixed points of a finite family of demi-contractive operators. Under suitable assumptions, we prove the strong convergence of this algorithm in the framework of a Hilbert space. The results obtained in this paper extend and improve the results of Tian and Jiang (2017), of Censor, Gibali and Reich (2012), and of many others. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new algorithm for solving a bilevel equilibrium problem in a real Hilbert space. In contrast to most other projection-type algorithms, which require to solve subproblems at each iteration, the subgradient method proposed in this paper requires only to calculate, at each iteration, two subgradients of convex functions and one projection onto a convex set. Hence, our algorithm has a low computational cost. We prove a strong convergence theorem for the proposed algorithm and apply it for solving the equilibrium problem over the fixed point set of a nonexpansive mapping. Some numerical experiments and comparisons are given to illustrate our results. Also, an application to Nash-Cournot equilibrium models of a semioligopolistic market is presented. [ABSTRACT FROM AUTHOR]

The fundamental phenomenon that has been used to enhance the convergence speed of learning automata (LA) is that of incorporating the running maximum likelihood (ML) estimates of the action reward probabilities into the probability updating rules for selecting the actions. The frontiers of this field have been recently expanded by replacing the ML estimates with their corresponding Bayesian counterparts that incorporate the properties of the conjugate priors. These constitute the Bayesian pursuit algorithm (BPA), and the discretized Bayesian pursuit algorithm. Although these algorithms have been designed and efficiently implemented, and are, arguably, the fastest and most accurate LA reported in the literature, the proofs of their $$\epsilon$$ -optimal convergence has been unsolved. This is precisely the intent of this paper. In this paper, we present a single unifying analysis by which the proofs of both the continuous and discretized schemes are proven. We emphasize that unlike the ML-based pursuit schemes, the Bayesian schemes have to not only consider the estimates themselves but also the distributional forms of their conjugate posteriors and their higher order moments-all of which render the proofs to be particularly challenging. As far as we know, apart from the results themselves, the methodologies of this proof have been unreported in the literature-they are both pioneering and novel. [ABSTRACT FROM AUTHOR]

In this paper, we propose an $$l_0$$ -norm penalized shrinkage linear affine projection ( $$l_0$$ -SL-AP) algorithm and an $$l_0$$ -norm penalized shrinkage widely linear affine projection ( $$l_0$$ -SWL-AP) algorithm. The proposed algorithms provide variable step-size by minimizing the noise-free a posteriori error at each iteration and introduce an $$l_0$$ -norm constraint to the cost function. The $$l_0$$ -SWL-AP algorithm also exploits noncircular properties of the input signal. In contrast with conventional AP algorithms, the proposed algorithms increase the estimation accuracy for time-varying sparse system identification. A quantitative analysis of the convergence behavior for the $$l_0$$ -SWL-AP algorithm verifies the capabilities of the proposed algorithms. To reduce the complexity, we also introduce dichotomous coordinate descent (DCD) iterations to the proposed algorithms ( $$l_0$$ -SL-DCD-AP and $$l_0$$ -SWL-DCD-AP) in this paper. Simulations indicate that the $$l_0$$ -SL-AP and $$l_0$$ -SWL-AP algorithms provide faster convergence speed and lower steady-state misalignment than the previous APA-type algorithms. The $$l_0$$ -SL-DCD-AP and $$l_0$$ -SWL-DCD-AP algorithms perform similarly to their counterparts but with reduced complexity. [ABSTRACT FROM AUTHOR]

The bat algorithm (BA) is a recent heuristic optimization algorithm based on the echolocation behavior of bats. However, the bat algorithm tends to fall into local optima and its optimization results are unstable because of its low global exploration ability. To solve these problems, a novel bat algorithm based on an integration strategy (IBA) is proposed in this paper. Through the integration strategy, an appropriate operator is adaptively selected to perform global search, so that the global search ability of the IBA is improved. Furthermore, the IBA disturbs the local optimum through a linear combination of Gaussian functions with different variances to avoid becoming trapped in local optima. The IBA also updates the velocity equation with an adaptive weight to further balance the exploration and exploitation. Moreover, the global convergence of the IBA is proved based on the convergence criterion of a stochastic algorithm. The performance of the IBA is evaluated on CEC2013 benchmark functions and compared with that of the standard BA as well as several of its variants. The results show that the IBA is superior to other algorithms. [ABSTRACT FROM AUTHOR]

This paper studies the convergence of the stochastic algorithm of the modified Robbins–Monro form for a Markov random field (MRF), in which some of the nodes are clamped to be observed variables while the others are hidden ones. Based on the theory of stochastic approximation, we propose proper assumptions to guarantee the Hölder regularity of both the update function and the solution of the Poisson equation. Under these assumptions, it is proved that the control parameter sequence is almost surely bounded and accordingly the algorithm converges to the stable point of the log-likelihood function with probability  1. [ABSTRACT FROM AUTHOR]

In this paper, we propose a unified view of gradient-based algorithms for stochastic convex composite optimization by extending the concept of estimate sequence introduced by Nesterov. More precisely, we interpret a large class of stochastic optimization methods as procedures that iteratively minimize a surrogate of the objective, which covers the stochastic gradient descent method and variants of the incremental approaches SAGA, SVRG, and MISO/Finito/SDCA. This point of view has several advantages: (i) we provide a simple generic proof of convergence for all of the aforementioned methods; (ii) we naturally obtain new algorithms with the same guarantees; (iii) we derive generic strategies to make these algorithms robust to stochastic noise, which is useful when data is corrupted by small random perturbations. Finally, we propose a new accelerated stochastic gradient descent algorithm and a new accelerated SVRG algorithm that is robust to stochastic noise. [ABSTRACT FROM AUTHOR]

In this paper, an uncountable infinite family of nonlinear mappings are investigated. Strong convergence theorems of common solutions are established in a strictly convex and uniformly smooth Banach space which also has the Kadec-Klee property. The results obtained in this paper unify and improve many corresponding results announced recently. [ABSTRACT FROM AUTHOR]

This paper reviews recent studies on the Particle Swarm Optimization (PSO) algorithm. The review has been focused on high impact recent articles that have analyzed and/or modified PSO algorithms. This paper also presents some potential areas for future study. [ABSTRACT FROM AUTHOR]

This paper considers identification of nonlinear rational systems defined as the ratio of two nonlinear functions of past inputs and outputs. Despite its long history, a globally consistent identification algorithm remains illusive. This paper proposes a globally convergent identification algorithm for such nonlinear rational systems. To the best of our knowledge, this is the first globally convergent algorithm for the nonlinear rational systems. The technique employed is a two-step estimator. Though two-step estimators are known to produce consistent nonlinear least squares estimates if a N consistent estimate can be determined in the first step, how to find such a N consistent estimate in the first step for nonlinear rational systems is nontrivial and is not answered by any two-step estimators. The technical contribution of the paper is to develop a globally consistent estimator for nonlinear rational systems in the first step. This is achieved by involving model transformation, bias analysis, noise variance estimation, and bias compensation in the paper. Two simulation examples and a practical example are provided to verify the good performance of the proposed two-step estimator. [ABSTRACT FROM AUTHOR]

The null space pursuit (NSP) algorithm is an operator-based signal separation approach which separates a signal into a set of additive subcomponents using adaptively estimated operators and parameters. In this paper, a new operator termed complex-valued differential (CD) operator is proposed. Combining with the CD operator, this paper proposes NSP-CD algorithm to solve the CD operator-based signal separation problem. The NSP-CD algorithm can separate the multi-component signal into sum of amplitude-modulated and frequency-modulated (AM–FM) signals in the form of A ( t ) exp ( j ) ( ϕ ( t ) ) . The proposed NSP-CD algorithm has many advantages. Firstly, the proposed CD operator can ensure that the AM–FM signal totally lies in the null space of the operator rather than close to the null space that the original used operator may reach. Secondly, compared with the original NSP algorithm, our algorithm provides a more reasonable strategy to update the regularization parameter λ and the leakage factor γ . Finally, we have proved that the proposed algorithm has quadric convergence theoretically. Experiments on both synthetic and real-life signals demonstrate that the NSP-CD algorithm is more robust and effective than other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

For ensemble learning, how to select and combine the candidate classifiers are two key issues which influence the performance of the ensemble system dramatically. Random vector functional link networks (RVFL) without direct input-to-output links is one of suitable base-classifiers for ensemble systems because of its fast learning speed, simple structure and good generalization performance. In this paper, to obtain a more compact ensemble system with improved convergence performance, an improved ensemble of RVFL based on attractive and repulsive particle swarm optimization (ARPSO) with double optimization strategy is proposed. In the proposed method, ARPSO is applied to select and combine the candidate RVFL. As for using ARPSO to select the optimal base RVFL, ARPSO considers both the convergence accuracy on the validation data and the diversity of the candidate ensemble system to build the RVFL ensembles. In the process of combining RVFL, the ensemble weights corresponding to the base RVFL are initialized by the minimum norm least-square method and then further optimized by ARPSO. Finally, a few redundant RVFL is pruned, and thus the more compact ensemble of RVFL is obtained. Moreover, in this paper, theoretical analysis and justification on how to prune the base classifiers on classification problem is presented, and a simple and practically feasible strategy for pruning redundant base classifiers on both classification and regression problems is proposed. Since the double optimization is performed on the basis of the single optimization, the ensemble of RVFL built by the proposed method outperforms that built by some single optimization methods. Experiment results on function approximation and classification problems verify that the proposed method could improve its convergence accuracy as well as reduce the complexity of the ensemble system. [ABSTRACT FROM AUTHOR]

Our contribution in this paper is to propose an iterative algorithm which does not require prior knowledge of operator norm and prove strong convergence theorem for approximating a solution of split common fixed point problem of demicontractive mappings in a real Hilbert space. So many authors have used algorithms involving the operator norm for solving split common fixed point problem, but as widely known the computation of these algorithms may be difficult and for this reason, authors have recently started constructing iterative algorithms with a way of selecting the step-sizes such that the implementation of the algorithm does not require the calculation or estimation of the operator norm. We introduce a new algorithm for solving the split common fixed point problem for demicontractive mappings with a way of selecting the step-sizes such that the implementation of the algorithm does not require the calculation or estimation of the operator norm and then prove strong convergence of the sequence in real Hilbert spaces. Finally, we give some applications of our result and numerical example at the end of the paper. [ABSTRACT FROM AUTHOR]

In this paper, we propose Linesearch methods for solving a bilevel split variational inequality problem (BSVIP) involving a strongly monotone mapping in the upper-level problem and pseudomonotone mappings in the lower-level one. A strongly convergent algorithm for such a BSVIP is proposed and analyzed. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new approach for sampling from probability measures in, possibly, high-dimensional spaces. By perturbing the standard overdamped Langevin dynamics by a suitable Stratonovich perturbation that preserves the invariant measure of the original system, we show that accelerated convergence to equilibrium and reduced asymptotic variance can be achieved, leading, thus, to a computationally advantageous sampling algorithm. The new perturbed Langevin dynamics is reversible with respect to the target probability measure and, consequently, does not suffer from the drawbacks of the nonreversible Langevin samplers that were introduced in C.-R. Hwang et al. (1993) [1] and studied in, e.g., T. Lelièvre et al. (2013) [2] and A.B. Duncan et al. (2016) [3] , while retaining all of their advantages in terms of accelerated convergence and reduced asymptotic variance. In particular, the reversibility of the dynamics ensures that there is no oscillatory transient behaviour. The improved performance of the proposed methodology, in comparison to the standard overdamped Langevin dynamics and its nonreversible perturbation, is illustrated on an example of sampling from a two-dimensional warped Gaussian target distribution. [ABSTRACT FROM AUTHOR]

Abstract We analyze a new random algorithm for numerical integration of d -variate functions over [ 0 , 1 ] d from a weighted Sobolev space with dominating mixed smoothness α ≥ 0 and product weights 1 ≥ γ 1 ≥ γ 2 ≥ ⋯ > 0 , where the functions are continuous and periodic when α > 1 ∕ 2. The algorithm is based on rank-1 lattice rules with a random number of points n. For the case α > 1 ∕ 2 , we prove that the algorithm achieves almost the optimal order of convergence of O (n − α − 1 ∕ 2) , where the implied constant is independent of the dimension d if the weights satisfy ∑ j = 1 ∞ γ j 1 ∕ α < ∞. The same rate of convergence holds for the more general case α > 0 by adding a random shift to the lattice rule with random n. This shows, in particular, that the exponent of strong tractability in the randomized setting equals 1 ∕ (α + 1 ∕ 2) , if the weights decay fast enough. We obtain a lower bound to indicate that our results are essentially optimal. This paper is a significant advancement over previous related works with respect to the potential for implementation and the independence of error bounds on the problem dimension. Other known algorithms which achieve the optimal error bounds, such as those based on Frolov's method, are very difficult to implement especially in high dimensions. Here we adapt a lesser-known randomization technique introduced by Bakhvalov in 1961. This algorithm is based on rank-1 lattice rules which are very easy to implement given the integer generating vectors. A simple probabilistic approach can be used to obtain suitable generating vectors. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we present a two-grid mixed finite element scheme for distributed optimal control problems governed by stationary Stokes equations. In order to avoid the difficulty caused by the symmetry constraint of the stress tensor, we use pseudostress to replace it. The state and co-state are approximated by the lowest order Raviart–Thomas mixed finite element spaces and the control variable is approximated by piecewise constant functions. We first prove that the difference between the interpolation and the numerical solution has superconvergence property for the control u with order h 2. Then, using the postprocessing technique, we derive a second-order superconvergent result for the control u . Next, we construct a two-grid mixed finite element scheme and derive a priori error estimates. Finally, a numerical experiment is presented to verify the theoretical results. [ABSTRACT FROM AUTHOR]

We present an infeasible interior-point predictor-corrector algorithm, based on a large neighborhood of the central path, for horizontal linear complementarity problem over the Cartesian product of symmetric cones. Throughout the paper, we assume that a certain property holds for the above-mentioned problem. This condition is equivalent to the property of sufficiency for the particular case of horizontal linear complementarity problem. The polynomial convergence is shown for the commutative class of search directions. We specialize our algorithm further by prescribing some scaling elements and also consider the case of feasible starting points. We believe this to be the first interior-point method based on large neighborhoods for the problem in consideration. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we employ matrix LSQR algorithm to deal with quaternionic least squares problem in order to find the minimum norm solutions with kinds of special structures, and propose a strategy to accelerate convergence rate of the algorithm via right–left preconditioning of the coefficient matrices. We mainly focus on analyzing the minimum norm η -Hermitian solution and the minimum norm η -biHermitian solution to the quaternionic least squares problem, η ∈ { i , j , k }. Other structured solutions also can be obtained using the proposed technique. A number of numerical experiments are performed to show the efficiency of the preconditioned matrix LSQR algorithm. [ABSTRACT FROM AUTHOR]

Abstract In the paper, we present an efficient two grid method for the miscible displacement problem which discretized by mixed finite element methods for the pressure equation and concentration equation at the same time, and then analyzed the error estimate of the two-gird algorithm. At last, the numerical experiment presented confirmed the theoretical results. Compared with the standard mixed finite element methods, this two-grid scheme based on the mixed methods can keep the same convergence order and cost much less work. [ABSTRACT FROM AUTHOR]

In Yao (J. Sci. Comput. 66, 19-40 2016), two Ljusternik-Schnirelman minimax algorithms for capturing multiple free saddle points are developed from well-known Ljusternik-Schnirelman critical point theory, numerical experiment is carried out and global convergence is established. In this paper, a Ljusternik-Schnirelman minimax algorithm for calculating multiple equality constrained saddle points is presented. The algorithm is applied to numerically solve eigen problems. Finally, global convergence for the algorithm is verified. [ABSTRACT FROM AUTHOR]

An R2 indicator-based multi-objective particle swarm optimiser (R2-MOPSO) can obtain well-convergence and well-distributed solutions while solving two and three objectives optimisation problems. However, R2-MOPSO faces difficulty to tackle many-objective optimisation problems because balancing convergence and diversity is a key issue in high-dimensional objective space. In order to address this issue, this paper proposes a novel algorithm, named R2-MaPSO, which combines the R2 indicator and decomposition-based archiving pruning strategy into particle swarm optimiser for many-objective optimisation problems. The innovations of the proposed algorithm mainly contains three crucial factors: (1) A bi-level archiving maintenance approach based on the R2 indicator and objective space decomposition strategy is designed to balance convergence and diversity. (2) The global-best leader selection is based on the R2 indicator and the personal-best leader selection is based on the Pareto dominance. Meanwhile, the objective space decomposition leader selection adopts the feedback information from the bi-level archive. (3) A new velocity updated method is modified to enhance the exploration and exploitation ability. In addition, an elitist learning strategy and a smart Gaussian learning strategy are embedded into R2-MaPSO to help the algorithm jump out of the local optimal front. The performance of the proposed algorithm is validated and compared with some algorithms on a number of unconstraint benchmark problems, i.e. DTLZ1-DTLZ4, WFG test suites from 3 to 15 objectives. Experimental results have demonstrated a better performance of the proposed algorithm compared with several multi-objective particle swarm optimisers and multi-objective evolutionary algorithms for many-objective optimisation problems. [ABSTRACT FROM AUTHOR]