Search

With the rapid development of renewable energy and the increasing maturity of energy storage technology, microgrids are quickly becoming popular worldwide. The stochastic scheduling problem of microgrids can increase operational costs and resource wastage. In order to reduce operational costs and optimize resource utilization efficiency, the real-time scheduling of microgrids becomes particularly important. After collecting extensive data, reinforcement learning (RL) can provide good strategies. However, it cannot make quick and rational decisions in different environments. As a method with generalization ability, meta-learning can compensate for this deficiency. Therefore, this paper introduces a microgrid scheduling strategy based on RL and meta-learning. This method can quickly adapt to different environments with a small amount of training data, enabling rapid energy scheduling policy generation in the early stages of microgrid operation. This paper first establishes a microgrid model, including components such as energy storage, load, and distributed generation (DG). Then, we use a meta-reinforcement learning framework to train the initial scheduling strategy, considering the various operational constraints of the microgrid. The experimental results show that the MAML-based RL strategy has advantages in improving energy utilization and reducing operational costs in the early stages of microgrid operation. This research provides a new intelligent solution for microgrids' efficient, stable, and economical operation in their initial stages. [ABSTRACT FROM AUTHOR]

This paper proposes a novel cloud computingbased parallel deep reinforcement learning (DRL) energy management strategy (EMS) for connected plug-in hybrid vehicles. First, a proximal policy optimization (PPO) algorithm is developed. Since the cloud computing can reduce the computational burden of the connected vehicles, the PPO is deployed in the proposed cloud computing-based EMS. In order to improve the strategy adaptation, a parallel mechanism is proposed to achieve the information interaction with multiple vehicles. Considering the real-time control requirements, thread pool is proposed and applied in the cloud computing based parallel EMS. The thread pool-based strategy provides an efficient real-time control ability and strategy improvement solution. To verify the PPO based EMS, the dynamic programming, deep Q-network and double deep Q-network strategies are developed for comparison. It is found that the PPO can achieve similar fuel efficiency improvement with the DP strategy among the three DRL algorithms. For parallel training of multiple connected vehicles, the cloud computing-based parallel EMS improves fuel economy by approximately 7.7%. Threadpool based parallel real-time EMS reduces average time for computational interactions by 20% and further improves the fuel efficiency. The proposed strategy has the advantages of realtime control, adaptability and continuous learning for improved fuel efficiency. [ABSTRACT FROM AUTHOR]

The Smart Grid AMI is at the forefront of modernizing energy management for efficient energy distribution and maintaining grid reliability. However, the seamless flow of data within AMI introduces critical security challenges, necessitating innovative approaches to ensure secure data transmission. This paper addresses these challenges by proposing a Quantum Key Distribution (QKD) based on Multi-Objective Optimization (MOO) and Reinforcement Learning (RL) framework for secure data transmission within the Smart Grid AMI. Primarily, we formulate the routing problem as a MOO challenge with four primary objectives namely, energy efficiency, latency minimization, reliability, and security to capture the essence of efficient and secure data transmission. Further, we introduce an RL agent based on the Proximal Policy Optimization (PPO) algorithm for robust policy learning. The RL agent by exploring diverse routing actions and receiving rewards optimizes the routing decisions within the network environment based on the objectives specified in the multi-objective function (MOF). A novel MOF quantifies trade-offs between security and performance metrics, integrating QKD-based security metrics with traditional optimization objectives. This function guides the RL agent in making informed routing decisions. Through extensive simulations using the NS-2 simulator in a realistic Smart Grid AMI environment, the proposed approach obtained energy consumption of 750 J, latency of 55 ms, and security level of 96% and also revealed significant improvements compared to conventional techniques widely employed for secured communication in AMI networks. Overall, the simulation results exhibited that the proposed method showed outstanding performance by achieving energy efficiency of 0.95, latency reduction of 0.92, reliability improvement of 0.94, and security enhancement of 0.96. Overall, this integration offers a pioneering solution to address the evolving security challenges in the Smart Grid landscape and contributes to the advancement of secure data transmission in Smart Grid AMI networks, fostering a resilient and secure energy ecosystem. [ABSTRACT FROM AUTHOR]

The service life and fuel consumption of fuel cell system (FCS) are the main factors limiting the commercialization of fuel cell electric vehicles (FCEV). Effective energy management strategies (EMS) can reduce fuel consumption during the cycle and prolong the service life of FCS. This paper proposes an energy management strategy based on the deep reinforcement learning (DRL) algorithm, deep Q-learning (DQL). Considering the unstable performance of conventional DQL during the training process, a new algorithm called Double Deep Q Learning (DDQL) is introduced. The DDQL uses a target evaluation network to evaluate output actions and a delayed update strategy to improve the convergence and stability of DRL. This article trains the strategy using UDDS cycle, tests it using combined cycles UDDS-WLTC-NEDC, and compares it with traditional ECM-based EMS. The results demonstrate that under the combined cycle, the strategy effectively reduced FCS voltage degradation by 50%, maintained fuel economy, and ensured consistency between the initial and final state of charge (SOC) of LIB. [ABSTRACT FROM AUTHOR]

• TIP-EMS enhances EV energy efficiency by integrated prediction and energy management. • Significant driving economy boost; 1.6%-4.2% better than state-of-the-art PEMS. • TIP-EMS shows robust performance with varying prediction weight adjustments. To enhance the energy efficiency of electrified vehicles (EVs), developing effective energy management strategies (EMS) for hybrid storage systems is essential. Predictive EMS (PEMS) that foresee future vehicle speeds have demonstrated substantial potential in boosting EMS performance. However, traditional PEMS models, employing a sequential approach of speed prediction followed by energy allocation, are hampered by cumulative errors. These errors from the initial speed predss this issue, this paper introduces a novel solution: the trainable integrated preiction negatively impact the efficiency of subsequent energy management. To addrediction and energy management strategy (TIP-EMS). Contrasting with conventional sequential PEMS, TIP-EMS features a dual-branch, integrated neural network, which is fully trainable. This network processes driving status inputs via attention layers, with one branch dedicated to energy management objectives using a reinforcement learning (RL) algorithm, and the other to vehicle speed prediction. Both branches are trained simultaneously, but post-training, only the RL branch is activated for energy management. Implemented with a soft actor-critic RL algorithm, TIP-EMS is applied to a fuel cell EV for optimized energy management. The validation involved training TIP-EMS using 27 driving profiles, which developed its prediction and energy management capabilities, followed by tests in untrained scenarios. The results show that TIP-EMS surpasses conventional sequential PEMS by up to 4.2% in scenarios where prediction accuracies are comparable, highlighting the efficacy of the trainable integrated mechanism. In addition, TIP-EMS exhibits superior energy conservation compared to non-predictive RL strategies. Lastly, TIP-EMS exhibits robustness to adjustments in the weight given to the prediction objective, further confirming its practical applicability. [ABSTRACT FROM AUTHOR]