Search

• A review is conducted on the application of RL algorithms for EV charging scheduling. • 111 relevant articles are analyzed from multiple perspectives. • Evidence from the literature reveals that RL is increasingly contributing to the field of EV charging scheduling. • Several research challenges and possible future research directions are identified. As climate change and environmental concerns have become increasingly pressing issues, electric vehicles (EVs) have emerged as a viable and environmentally-friendly alternative to their traditional gasoline-powered counterparts. With the growing popularity of EVs, it is widely recognized that scheduled EV charging is of vital importance in enhancing users' satisfaction, improving the profitability of charging stations, and ensuring the secure operation of power grids. Therefore, effective charging scheduling is crucial to achieve EV integration into modern mobility and further promote mass EV adoption. However, EV charging scheduling problems (CSPs) present a significant challenge to conventional methods owing to the complex nature of the decision environment. Reinforcement learning (RL) has gained considerable attention for addressing various CSPs with a Markov decision process (MDP) formulation. This paper aims to conduct a systematic review of existing studies that utilize RL to tackle CSPs. Firstly, we present a summary of the state-of-the-art CSPs and RL algorithms, followed by a review of previous research studies and a discussion of the current challenges in this field. Finally, several unresolved issues and possible future research topics are identified based on our review findings. [ABSTRACT FROM AUTHOR]

Electric vehicles (EVs) have received considerable attention in dealing with severe environmental and energy crises. The capacity planning of public charging stations has been a major factor in facilitating the wide market penetration of EVs. In this paper, we present an optimization model for charging station capacity planning to maximize the fuzzy quality of service (FQoS) considering queuing behavior, blocking reliability, and multiple charging options classified by battery technical specifications. The uncertainty of the EV arrival and service time are taken into account and described as fuzzy numbers characterized by triangular membership functions. Meanwhile, an $\alpha$ -cuts-based algorithm is proposed to defuzzify the FQoS. Finally, the numerical results illustrate that a more robust plan can be obtained by accounting for FQoS. The contribution of the proposed model allows decision-makers and operators to plan the capacity of charging stations with fuzzy EV arrival rate and service rate and provide a better service for customers with different charging options. [ABSTRACT FROM AUTHOR]

This paper investigates the electric vehicle (EV) charging scheduling problem for public EV charging stations (EVCSs) that can accommodate heterogeneous charging demands, aiming to flatten the aggregate load on the power grid and reduce the peak demand. In contrast to existing works that mainly focus on a single scheduling strategy, a two-level hierarchical charging scheduling method is proposed, which includes an online booking system (OBS) and a pricing-based charging control system (PCCS). Specifically, the implementation of OBS can reduce the service capacity of bookable chargers, whereas PCCS can facilitate the redistribution of charging demand from peak to off-peak time periods. Moreover, the nonlinear charging profile of the battery is incorporated into the model to better reflect the real charging process. A deep reinforcement learning (DRL)-based approach with a discrete Markov decision process (MDP) formulation is designed to find the optimal scheduling solution, where deep Q-network (DQN) and deep deterministic policy gradient (DDPG) are combined to handle the hybrid action space with both discrete and continuous actions. A comprehensive set of experiments is carried out to examine the effectiveness of the developed two-level scheduling scheme. The results indicate that the proposed method improves the scheduling effectiveness compared to single-strategy methods by 6.7% to 49.44%. [ABSTRACT FROM AUTHOR]

This study addresses the optimal electric vehicle (EV) charging station deployment problem (CSDP) on coupled transportation and power distribution networks, which is one of the critical issues with the mass adoption of EVs in the recent years. In contrast to existing works that mainly employ heuristics and exact algorithms, we propose a finite-discrete Markov decision process (MDP) formulation defined in a reinforcement learning (RL) framework to mitigate the curse of dimensionality problem. The RL-based approach aims to determine the location of a set of EV charging stations with limited capacity by minimizing the total investment cost while satisfying the coupled network constraints. Specifically, a long short-term memory (LSTM)-based recurrent neural network (RNN) with an attention mechanism is used to train the model based on an offline strategy. The model parameters are learned by the policy gradient algorithm with a learned baseline function. Numerical experiments on multiple problem sizes are conducted to assess the efficiency and feasibility of the proposed solution method. We experimentally show that our approach is efficient to solve the CSDP and outperforms other baseline approaches in solution quality with competitive computational time. • We propose an RL framework to address the EV charging station deployment problem. • An RNN-based model is developed to find the optimal policy. • A policy gradient method is used to train the agent based on an offline strategy. • Experiments are conducted to test the performance of the proposed method. [ABSTRACT FROM AUTHOR]