Back to Search
Start Over
State-Temporal Compression in Reinforcement Learning With the Reward-Restricted Geodesic Metric.
- Source :
-
IEEE Transactions on Pattern Analysis & Machine Intelligence . Sep2022, Vol. 44 Issue 9, p5572-5589. 18p. - Publication Year :
- 2022
-
Abstract
- It is difficult to solve complex tasks that involve large state spaces and long-term decision processes by reinforcement learning (RL) algorithms. A common and promising method to address this challenge is to compress a large RL problem into a small one. Towards this goal, the compression should be state-temporal and optimality-preserving (i.e., the optimal policy of the compressed problem should correspond to that of the uncompressed problem). In this paper, we propose a reward-restricted geodesic (RRG) metric, which can be learned by a neural network, to perform state-temporal compression in RL. We prove that compression based on the RRG metric is approximately optimality-preserving for the raw RL problem endowed with temporally abstract actions. With this compression, we design an RRG metric-based reinforcement learning (RRG-RL) algorithm to solve complex tasks. Experiments in both discrete (2D Minecraft) and continuous (Doom) environments demonstrated the superiority of our method over existing RL approaches. [ABSTRACT FROM AUTHOR]
- Subjects :
- *REINFORCEMENT learning
*GEODESICS
*DECISION making
*SEMICONDUCTOR devices
Subjects
Details
- Language :
- English
- ISSN :
- 01628828
- Volume :
- 44
- Issue :
- 9
- Database :
- Academic Search Index
- Journal :
- IEEE Transactions on Pattern Analysis & Machine Intelligence
- Publication Type :
- Academic Journal
- Accession number :
- 158406110
- Full Text :
- https://doi.org/10.1109/TPAMI.2021.3069005