1. Residual Sarsa algorithm with function approximation.
- Author
-
Qiming, Fu, Wen, Hu, Quan, Liu, Heng, Luo, Lingyao, Hu, and Jianping, Chen
- Subjects
- *
APPROXIMATION algorithms , *REINFORCEMENT learning , *VECTOR valued functions , *BENCHMARK problems (Computer science) , *LEARNING , *MACHINE learning - Abstract
In this work, we proposed an efficient algorithm named the residual Sarsa algorithm with function approximation (FARS) to improve the performance of the traditional Sarsa algorithm, and we use the gradient-descent method to update the function parameter vector. In the learning process, the Bellman residual method is adopted to guarantee the convergence of the algorithm, and a new rule for updating vectors of action-value functions is adopted to solve unstable and slow convergence problems. To accelerate the convergence rate of the algorithm, we introduce a new factor, named the forgotten factor, which can help improve the robustness of the algorithm's performance. Based on two classical reinforcement learning benchmark problems, the experimental results show that the FARS algorithm has better performance than other related reinforcement learning algorithms. [ABSTRACT FROM AUTHOR]
- Published
- 2019
- Full Text
- View/download PDF