Search

When dealing with sparse reward problems,existing deep RL algorithms often lead to hard exploration,they often only rely on the pre-designed environment reward,so it is difficult to achieve good results.In this situation,it is necessary to design rewards more carefully,make more accurate judgments and feedback on the exploration status of agents.The asynchronous advantage actor-critic(A3C) algorithm improves the training efficiency through parallel training,and improves the training speed of the original algorithm.However,for the environment with sparse rewards,it cannot well solve the problem of difficult exploration.To solve the problem of poor exploration effect of A3C algorithm in sparse reward environment,A3C based on exploration trajectory perception(ETP-A3C) is proposed.The algorithm can perceive the exploration trajectory of the agent when it is difficult to explore in training,further judge and decide the exploration direction of the agent,and help the agent get out of the exploration dilemma as soon as possible.In order to verify the effectiveness of ETP-A3C algorithm,a comparative experiment is carried out with baseline algorithm in five different environments of Super Mario Brothers.The results show that this method has significantly improved the learning speed and model stability.

In the generation process of reinforcement learning decision model, due to the complex environment and incomplete observation of state information, the classical proximal policy optimization algorithm faces problems such as low exploration and utilization efficiency and poor effect of generated strategies, this paper proposes an MNAEK-PPO (proximal policy optimization based on maximum number of arrival & expert knowledge algorithm) based on curiosity mechanism. Focusing on the difficult problem of exploring the strategy space, by constructing the exploration frequency matrix of the agent in the training process, the exploration frequency is treated as an internal reward to participate in the agent's reinforcement learning and training process. In addition, expert knowledge is added to assist the agent in making decisions. Through experiments in the intelligent battlefield simulation environment, the best construction method of internal rewards in MNAEK-PPO is determined, and a series of comparative experiments are carried out. The experimental results show that MNAEK-PPO greatly improves the exploration efficiency of decision space, and the convergence speed and game score are significantly improved, which provides a new solution for promoting the application and development of deep reinforcement learning in the generation of intelligent tactical strategies. [ABSTRACT FROM AUTHOR]