1. Provable distributed adaptive temporal-difference learning over time-varying networks.
- Author
-
Zhu, Junlong, Li, Bing, Wang, Lin, Zhang, Mingchuan, Xing, Ling, Xi, Jiangtao, and Wu, Qingtao
- Subjects
- *
TIME-varying networks , *MACHINE learning , *MARL , *REINFORCEMENT learning , *PROBLEM solving , *ALGORITHMS , *DISTRIBUTED algorithms - Abstract
Multi-agent reinforcement learning (MARL) has been successfully applied in many fields. In MARL, the policy evaluation problem is one of crucial problems. In order to solve this problem, distributed Temporal-Difference (TD) learning algorithm is one of the most popular methods in a cooperative manner. Despite its empirical success, however, the theory of the adaptive variant of distributed TD learning still remain limited. To fill this gap, we propose an adaptive distributed temporal-difference algorithm (referred to as MS - ADTD) under Markovian sampling over time-varying networks. Furthermore, we rigorously analyze the convergence of MS - ADTD , the theoretical results show that the local estimation can converge linearly to the optimal neighborhood. Meanwhile, the theoretical results are verified by simulation experiments. • This paper proposes a distributed adaptive TD learning algorithm under Markovian sampling. • It theoretically analyzes the non-asymptotic convergence performance of the proposed algorithm. • It verifies the performance of the proposed algorithm by experiments. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF