Search

In this paper a new distributed multi-agent Actor-Critic algorithm for reinforcement learning is proposed for solving multi-agent multi-task optimization problems. The Critic algorithm is in the form of a Distributed Emphatic Temporal Difference DETD(λ) algorithm, while the Actor algorithm is proposed as a complementary consensus based policy gradient algorithm, derived from a global objective function having the role of a scalarizing function in multi-objective optimization. It is demonstrated that the Feller-Markov properties hold for the newly derived Actor algorithm. A proof of the weak convergence of the algorithm to the limit set of an attached ODE is derived under mild conditions, using a specific decomposition between the Critic and the Actor algorithms and additional two-time-scale stochastic approximation arguments. An experimental verification of the algorithm properties is given, showing that the algorithm can represent an efficient tool for practice. [ABSTRACT FROM AUTHOR]

In this paper an approach is proposed to decentralized multi-agent identification of large-scale systems represented by linear discrete-time stochastic MIMO models. It is assumed that each agent: (a) has access only to a subset of noisy input–output variables; (b) communicates local data processing results to its neighborhood. The proposed algorithm consists of two stages. The first stage is a consensus-based stochastic approximation algorithm for estimating input–output correlation functions, while at the second stage each agent utilizes a stochastic approximation algorithm with expanding truncations derived from the modified Yule–Walker equations in order to generate all the system parameter estimates. It is proved that under nonrestrictive assumptions concerning the system properties and the multi-agent network topology the estimates of the correlation functions converge almost surely to their true values and those of the system parameters to a solution of the modified Yule–Walker equations, assuming intermittent observations and communication outages. Conditions are also given for the strong consistency of the parameter estimates. Simulation results provide an illustration of the algorithm properties. [ABSTRACT FROM AUTHOR]

Abstract: In this paper a new algorithm for discrete-time overlapping decentralized state estimation of large scale systems is proposed in the form of a multi-agent network based on a combination of local estimators of Kalman filtering type and a dynamic consensus strategy, assuming intermittent observations and communication faults. Under general conditions concerning the agent resources and the network topology, conditions are derived for the convergence to zero of the estimation error mean and for the mean-square estimation error boundedness. A centralized strategy based on minimization of the steady-state mean-square estimation error is proposed for selection of the consensus gains; these gains can also be adjusted by local adaptation schemes. It is also demonstrated that there exists a connection between the network complexity and efficiency of denoising, i.e., of suppression of the measurement noise influence. Several numerical examples serve to illustrate characteristic properties of the proposed algorithm and to demonstrate its applicability to real problems. [Copyright &y& Elsevier]

In this paper we propose a novel stochastic consensus seeking algorithm based on the introduction of a nonlinear transformation aimed at robustification with respect to noise influence. The introduced nonlinear transformation is selected according to the methodology of stochastic approximation and robust statistics. The proposed algorithm represents a general nonlinear stochastic consensus seeking scheme, not yet treated in the literature. It provides a significant improvement over the linear algorithms from the point of view of robustness to noise, ensuring better convergence rate and lower sensitivity of the limit state value at consensus. One of the main contributions of the paper is the proof that the algorithm converges almost surely to consensus under general conditions. A detailed analysis of the limit state value at consensus is provided together with an insight into achievable convergence rate. Illustrative simulation results are also provided, demonstrating great advantages of the proposed algorithm compared to the existing consensus schemes. [ABSTRACT FROM AUTHOR]