Efficient hierarchical policy network with fuzzy rules

Hierarchical reinforcement learning (HRL) is a promising method, which decomposes complex tasks into a series of sub-tasks. However, at present, most HRL methods have slow convergence speed and are difficult to be widely applied to such scenarios in real life. In this paper, we propose an efficient hierarchical reinforcement learning algorithm with fuzzy rules (HFR), a novel framework for integrating human prior knowledge with hierarchical policy network, which can effectively accelerate the optimization of policy. The model presented in this paper uses fuzzy rules to represent the human prior knowledge, making the rules trainable because of the derivability of the fuzzy rules. In addition, a switch module that adaptively adjusts the decision-making frequency of the upper-level policy is proposed to solve the limitation of manual tuning. Experiment results demonstrate that HFR has a faster convergence rate than the current state-of-the-art HRL algorithms, especially in complex scenarios, such as robot control tasks.