Multi-agent energy management optimization for integrated energy systems under the energy and carbon co-trading market.

• An IESs co-trading market with carbon trading mechanisms is proposed. • The parameters and solving process are redesigned with the improved MADDPG algorithm. • The improved algorithm achieves fair trade and entity privacy protection among multiple entities. • The optimization strategy guides IESs to carry out "energy-carbon" joint management. With the development trends of carbon neutrality, carbon trading is graduating embedded into the energy management of integrated energy systems (IESs). The dual benefices of carbon emission reduction and economics can be achieved by coordinatively optimizing the complementarity and flexibility between multiple IESs. However, this leads to the increased complexity of the market transactions, which poses significant challenges in terms of the benefits distribution among multiple entities, the convergence of trading processes, and the privacy-preserving issues. The multi-agent reinforcement learning (MARL) is capable of solving complex sequential-decision problems and acquiring the optimal strategies for each entity through the interactions between the multiple agents and the market. The MARL deployed on the local agent can provide online trading decisions for individual market entities considering their own interests, which offers new potential to solve the abovementioned difficulties. In this paper, we proposed an IESs co-trading market including electricity, natural gas and carbon trading. The multi-agent energy management coordinative optimization problem is solved by an improved Multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm to achieve fair trade and entity privacy protection. The case study results verify that the proposed optimal energy management strategy based on the improved MADDPG algorithm can efficiently guide the IESs in the energy and carbon co-trading market. [ABSTRACT FROM AUTHOR]