Search

Abstract Local integrated energy system (IES), usually a multi‐building heating and cooling system incorporating cogeneration systems and distributed energy resources (DERs), is becoming an efficient energy infrastructure for energy system decarbonization. However, for small and medium‐scale local IESs, the fluctuation of user loads seriously influences energy system operation, and the increasing DER penetration also enlarges the influence. Given the necessity of exploring an efficient way to handle the uncertainties in local IES, this paper proposes a reinforcement learning (RL) approach based on the improved TD3 algorithm. The mathematical model of local IES is first established considering supply‐ and load‐side flexible resources. The local IES dispatch problem is formulated as a Markov decision process (MDP), in which multi‐type uncertainties of renewable generation, electric load and heat load are considered. For solving the MDP, an improved twin delayed deep deterministic policy gradient (TD3) algorithm is proposed with a dynamic balance mechanism of exploration noise. Based on a local IES testbed in the Nantong Central Innovation District, China, a comparison analysis is conducted to verify the promoting effect of flexible resources on the operation economy and renewable energy consumption. The system operating cost reduces by 18.46%, and surplus renewable energy can all be accommodated considering flexible resources. The dispatch policies obtained by the deep deterministic policy gradient (DDPG), the improved TD3, the original TD3 and traditional optimization algorithms are also compared. The results reveal that the convergence stability and solving accuracy of the improved TD3 outperform the other two RL algorithms. Specifically, the system operating cost of the improved TD3 reduces by 2.76% compared with the DDPG, and the energy supply imbalance decreases by around 88%. Meanwhile, the improved TD3 exhibits better operation economy and adaptability to the uncertain environment than the deterministic optimization and intraday rolling algorithms.

Abstract The development of hydrogen‐enriched compressed natural gas (HCNG) can make full use of the existing natural gas infrastructure, thereby alleviating the economic and technical pressure on the development of pure hydrogen. This paper proposes a multi‐stage flexible planning method for the regional electricity‐HCNG‐integrated energy system (E‐HCNG‐IES), which considers the medium and long‐term dynamic characteristics of the E‐HCNG‐IES. For each sub‐regional energy station, an energy‐hub type model including hydrogen/HCNG‐related equipment and facilities is established. Considering the influence of hydrogen blending on the energy delivery capacity and line pack of pipelines, a hydrogen‐resistant retrofit model for existing natural gas pipelines between sub‐regions is proposed. Meanwhile, the model for newly constructed hydrogen pipelines is also presented. Subsequently, a series of flexibility evaluation indicators are proposed to support the performance analysis of planning schemes. The lateral comparison of sub‐cases in Changzhou, China demonstrates that considering the hydrogen‐resistant retrofit of natural gas pipelines can reduce the construction cost of hydrogen pipelines by about 37.12%. The peak shaving capacity of underground hydrogen storage (UHS), the energy recycling ability of turbo expander (TE), and the space‐time decoupling ability of hydrogen in energy supply and demand have also been confirmed.