Back to Search Start Over

Informed POMDP: Leveraging Additional Information in Model-Based RL

Authors :
Lambrechts, Gaspard
Bolland, Adrien
Ernst, Damien
Publication Year :
2023

Abstract

In this work, we generalize the problem of learning through interaction in a POMDP by accounting for eventual additional information available at training time. First, we introduce the informed POMDP, a new learning paradigm offering a clear distinction between the information at training and the observation at execution. Next, we propose an objective that leverages this information for learning a sufficient statistic of the history for the optimal control. We then adapt this informed objective to learn a world model able to sample latent trajectories. Finally, we empirically show a learning speed improvement in several environments using this informed world model in the Dreamer algorithm. These results and the simplicity of the proposed adaptation advocate for a systematic consideration of eventual additional information when learning in a POMDP using model-based RL.<br />Comment: In Reinforcement Learning Conference, 2024. 10 pages, 22 pages total, 10 figures

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2306.11488
Document Type :
Working Paper