Search

Abstract Most mobile battery energy storage systems (MBESSs) are designed to enhance power system resilience and provide ancillary service for the system operator using energy storage. As the penetration of renewable energy and fluctuation of the electricity price increase in the power system, the demand‐side commercial entities can be more profitable utilizing the mobility and flexibility of MBESSs compared to the stational energy storage system. The profit is closely related to the spatiotemporal decision model and is influenced by environmental uncertainties, such as electricity price and traffic conditions. However, solving the real‐time control problem considering long‐term profit and uncertainties is time‐consuming. To address this problem, this paper proposes a deep reinforcement learning framework for MBESSs to maximize profit through market arbitrage. A knowledge‐assisted double deep Q network (KA‐DDQN) algorithm is proposed based on such framework to learn the optimal policy and increase the learning efficiency. Moreover, two criteria action generation methods of knowledge‐assisted learning are proposed for integer actions utilizing scheduling and short‐term programming results. Simulation results show that the proposed framework and method can achieve the optimal result, and KA‐DDQN can accelerate the learning process compared to the original method by approximately 30%.

Due to the advantages of their drive configuration form, skid-steering vehicles with independent wheel drive systems are widely used in various special applications. However, obtaining a reasonable distribution of the driving torques for the coordinated control of independent driving wheels is a challenging problem. In this paper, we propose a torque distribution strategy based on the Knowledge-Assisted Deep Deterministic Policy Gradient (KA-DDPG) algorithm, in order to minimize the desired value tracking error as well as achieve the longitudinal speed and yaw rate tracking control of skid-steering vehicles. The KA-DDPG algorithm combines knowledge-assisted learning methods with the DDPG algorithm, within the framework of knowledge-assisted reinforcement learning. To accelerate the learning process of KA-DDPG, two assisted learning methods are proposed: a criteria action method and a guiding reward method. The simulation results obtained, considering different scenarios, demonstrate that the KA-DDPG-based torque distribution strategy allows a skid-steering vehicle to achieve high performance, in tracking the desired value. In addition, further simulation results, also, demonstrate the contributions of knowledge-assisted learning methods to the training process of KA-DDPG: the criteria action method speeds up the learning speed by reducing the agent's random action selection, while the guiding reward method achieves the same result by sharpening the reward function. [ABSTRACT FROM AUTHOR]

This article studies the pump scheduling optimization problem in water distribution networks (WDNs) through a novel algorithm that combines knowledge learning and deep reinforcement learning. The optimization problem is modeled as a Markov decision process by taking three objectives in pressure management into consideration. Knowledge-assisted learning is incorporated into the reinforcement learning framework (KA-RL) to help evaluate the state value and guide the design of reward function, since the proposed KA-RL framework leverages the notion that historical data of WDNs could be utilized to produce optimal trajectories with respect to parametric variations. The knowledge-assisted proximal policy optimization (KA-PPO) algorithm, which only uses nodal pressure data, is proposed based on the KA-RL framework to address the arbitrary WDN topology and time-varying water demand. The effectiveness and applicability of the proposed algorithm are illustrated by virtue of the 22-node network with two pumps in a pump station. Empirical results demonstrate that KA-PPO works well in practice and compares favorably to Nelder–Mead method. [ABSTRACT FROM AUTHOR]

Due to the advantages of their drive configuration form, skid-steering vehicles with independent wheel drive systems are widely used in various special applications. However, obtaining a reasonable distribution of the driving torques for the coordinated control of independent driving wheels is a challenging problem. In this paper, we propose a torque distribution strategy based on the Knowledge-Assisted Deep Deterministic Policy Gradient (KA-DDPG) algorithm, in order to minimize the desired value tracking error as well as achieve the longitudinal speed and yaw rate tracking control of skid-steering vehicles. The KA-DDPG algorithm combines knowledge-assisted learning methods with the DDPG algorithm, within the framework of knowledge-assisted reinforcement learning. To accelerate the learning process of KA-DDPG, two assisted learning methods are proposed: a criteria action method and a guiding reward method. The simulation results obtained, considering different scenarios, demonstrate that the KA-DDPG-based torque distribution strategy allows a skid-steering vehicle to achieve high performance, in tracking the desired value. In addition, further simulation results, also, demonstrate the contributions of knowledge-assisted learning methods to the training process of KA-DDPG: the criteria action method speeds up the learning speed by reducing the agent’s random action selection, while the guiding reward method achieves the same result by sharpening the reward function.

Cooperative wind farm control is a complex problem due to wake effect, and it is hard to find the proper model. Reinforcement learning can find the optimal policy in a dynamic environment using “trial and error,” but may damage the machine and cause high cost during the learning process. In order to address this challenge, this article proposes the knowledge-assisted reinforcement learning framework by combining the low-fidelity analytical model with a reinforcement learning framework. Moreover, the knowledge-assisted deep deterministic policy gradient (KA-DDPG) algorithm and three kinds of knowledge-assisted learning methods are proposed based on the framework. The proposed methods are tested in nine different scenarios of WFSim. The simulation results show that the KA-DDPG algorithm can reach the maximum power output and ensure safety during learning. In addition, the learning cost is reduced by accelerating the learning process. [ABSTRACT FROM AUTHOR]