Memory-efficient distribution-guided experience sampling for policy consolidation.

• Explicitly modeling the distribution of transitions with a distributional neural network. • Selecting transitions with a novel distribution-guided experience sampling approach. • Consolidating the policy with most informative transitions in a memory-efficient way. • Performance of our approach is demonstrated on multiple control tasks. Policy consolidation aims at learning new skills in sequential multi-tasks without forgetting acquired skills. Typically, the collected transitions for each task need to be stored for further exploiting. However, it is impractical to store all the transitions with limited memory as the number of tasks increases. To this end, we propose a novel distribution-guided experience sampling method that can efficiently select previously informative transitions and replay them to consolidate the learned policy. Specifically, we learn a distributional neural network to capture the interdependence among transitions, and evaluate the importance of each transition based on the interdependence. Meanwhile, the importance-based prioritized sampling method is presented to periodically replay transitions with higher importance to consolidate previous skills. Therefore, the proposed method can maintain good generalization ability of the policy in a memory-efficient way. Experimental results demonstrate the effectiveness of the proposed method on several benchmarks. [ABSTRACT FROM AUTHOR]