Model-Free Real-Time Autonomous Control for a Residential Multi-Energy System Using Deep Reinforcement Learning

Multi-energy systems (MES) are attracting increasing attention driven by its potential to offer significant flexibility in future smart grids. At the residential level, the roll-out of smart meters and rapid deployment of smart energy devices call for autonomous multi-energy management systems which can exploit real-time information to optimally schedule the usage of different devices with the aim of minimizing end-users’ energy costs. This paper proposes a novel real-time autonomous energy management strategy for a residential MES using a model-free deep reinforcement learning (DRL) based approach, combining state-of-the-art deep deterministic policy gradient (DDPG) method with an innovative prioritized experience replay strategy. This approach is tailored to align with the nature of the problem by posing it in multi-dimensional continuous state and action spaces, facilitating more cost-effective control strategies to be devised. The superior performance of the proposed approach in reducing end-user’s energy cost while coping with the MES uncertainties is demonstrated by comparing it against state-of-the-art DRL methods as well as conventional stochastic programming and robust optimization methods in numerous case studies in a real-world scenario.