Search

To address the issues of exploration imbalance and slow convergence speed in the Q‐learning path planning algorithm, an adaptive improved Q‐learning path planning algorithm based on an obstacle learning matrix and artificial potential field (APF) is proposed. First, an obstacle learning matrix is established to store the positions of obstacles encountered in each learning iteration, avoiding redundant learning of the same obstacle. Second, an adaptive exploration enhancement strategy is introduced by incorporating the concept of success rate. This strategy divides the decay process of the exploration rate into an exploration‐dominant initial stage and an exploitation‐dominant later stage. Finally, an APF‐weighted action selection strategy is introduced, utilizing the guiding force generated by the APF to encourage the mobile robot to avoid obstacles more efficiently. Simulation results show that the proposed method can effectively reduce the number of iterations and time consumption in the optimization process, resulting in a smoother and more stable planned path. [ABSTRACT FROM AUTHOR]

By integrating deep neural networks with reinforcement learning, the Double Deep Q Network (DDQN) algorithm overcomes the limitations of Q-learning in handling continuous spaces and is widely applied in the path planning of mobile robots. However, the traditional DDQN algorithm suffers from sparse rewards and inefficient utilization of high-quality data. Targeting those problems, an improved DDQN algorithm based on average Q-value estimation and reward redistribution was proposed. First, to enhance the precision of the target Q-value, the average of multiple previously learned Q-values from the target Q network is used to replace the single Q-value from the current target Q network. Next, a reward redistribution mechanism is designed to overcome the sparse reward problem by adjusting the final reward of each action using the round reward from trajectory information. Additionally, a reward-prioritized experience selection method is introduced, which ranks experience samples according to reward values to ensure frequent utilization of high-quality data. Finally, simulation experiments are conducted to verify the effectiveness of the proposed algorithm in fixed-position scenario and random environments. The experimental results show that compared to the traditional DDQN algorithm, the proposed algorithm achieves shorter average running time, higher average return and fewer average steps. The performance of the proposed algorithm is improved by 11.43% in the fixed scenario and 8.33% in random environments. It not only plans economic and safe paths but also significantly improves efficiency and generalization in path planning, making it suitable for widespread application in autonomous navigation and industrial automation. [ABSTRACT FROM AUTHOR]

Targeting some problems of the RRT_Connect path planning algorithm, such as average search and low efficiency, proposes an improved RRT_Connect algorithm that may optimize the searched nodes and parts of planned paths. Firstly, an improved RRT_Connect algorithm based on destination and searched node bias strategy is proposed. Secondly, an improved RRT_Connect algorithm is put forward for the optimization of the searched nodes and some planned paths to deal with the problem of low quality reflected in the improved RRT_Connect path planning algorithm, and the optimization for the cost of path planning by figuring out valid new nodes and parent nodes of adjacent nodes within a certain range. On this basis, the path planning algorithm is verified by simulation and actual experiments. It is shown by the experimental results that the improved RRT_Connect algorithm proposed in this paper can not only shorten the time and length of path planning but also decrease the number of search iterations and nodes. [ABSTRACT FROM AUTHOR]

Constrained by the numbers of action space and state space, Q-learning cannot be applied to continuous state space. Targeting this problem, the double deep Q network (DDQN) algorithm and the corresponding improvement methods were explored. First of all, to improve the accuracy of the DDNQ algorithm in estimating the target Q value in the training process, a multi-step guided strategy was introduced into the traditional DDQN algorithm, for which the single-step reward was replaced with the reward obtained in continuous multi-step interactions of mobile robots. Furthermore, an experience classification training method was introduced into the traditional DDQN algorithm, for which the state transition generated by the mobile robot–environment interaction was divided into two different types of experience pools, and experience pools were trained by the Q network, and the sampling proportions of the two experience pools were updated through the training loss. Afterward, the advantages of a multi-step guided DDQN (MS-DDQN) algorithm and experience classification DDQN (EC-DDQN) algorithm were combined to develop a novel experience classification multi-step DDQN (ECMS-DDQN) algorithm. Finally, the path planning of these four algorithms, including DDQN, MS-DDQN, EC-DDQN, and ECMS-DDQN, was simulated on the OpenAI Gym platform. The simulation results revealed that the ECMS-DDQN algorithm outperforms the other three in the total return value and generalization in path planning. [ABSTRACT FROM AUTHOR]

Directing at various problems of the traditional Q-Learning algorithm, such as heavy repetition and disequilibrium of explorations, the reinforcement-exploration strategy was used to replace the decayed ε-greedy strategy in the traditional Q-Learning algorithm, and thus a novel self-adaptive reinforcement-exploration Q-Learning (SARE-Q) algorithm was proposed. First, the concept of behavior utility trace was introduced in the proposed algorithm, and the probability for each action to be chosen was adjusted according to the behavior utility trace, so as to improve the efficiency of exploration. Second, the attenuation process of exploration factor ε was designed into two phases, where the first phase centered on the exploration and the second one transited the focus from the exploration into utilization, and the exploration rate was dynamically adjusted according to the success rate. Finally, by establishing a list of state access times, the exploration factor of the current state is adaptively adjusted according to the number of times the state is accessed. The symmetric grid map environment was established via OpenAI Gym platform to carry out the symmetrical simulation experiments on the Q-Learning algorithm, self-adaptive Q-Learning (SA-Q) algorithm and SARE-Q algorithm. The experimental results show that the proposed algorithm has obvious advantages over the first two algorithms in the average number of turning times, average inside success rate, and number of times with the shortest planned route. [ABSTRACT FROM AUTHOR]