Search

In this study, a decision-making and motion planning controller with continuous action space is constructed in the highway driving scenario based on deep reinforcement learning. In the decision-making and planning problem, the goal is to achieve the safety, efficiency, and comfort of automated vehicles. In the driving scenario, the surrounding vehicles are controlled by the intelligent driver model and a general model (minimizing overall braking induced by lane change, MOBIL), which enables them to react to the environment and mimic the vehicle interactions on the highway. Given the uncertainties in the driving conditions, a specific deep reinforcement learning technique, called soft actor-critic, is used to solve the decision-making and planning problem with continuous action space. Simulation results show that the proposed method can solve the decision-making and motion planning problem in the interactive traffic environment to carry out safe lane-change maneuvers and cruise at high speed. In addition, two control policies are developed with different weights on safety, efficiency, and comfort. [ABSTRACT FROM AUTHOR]

Knowledge transfer is a promising concept to achieve real-time decision-making for autonomous vehicles. This paper constructs a transfer deep reinforcement learning (RL) framework to transform the driving tasks in the intersection environments. The driving missions at the unsignalized intersection are cast into a left turn, right turn, and running straight for automated vehicles. The goal of the autonomous ego vehicle (AEV) is to drive through the intersection situation efficiently and safely. This objective promotes the studied vehicle to increase its speed and avoid crashing other vehicles. The decision-making policy learned from one driving task is transferred through three transfer rules in another driving mission and evaluated. Simulation results reveal that the decision-making strategies related to similar tasks are transferable and have a high success rate. It indicates that the presented control framework could reduce time consumption and realize online implementation. Therefore, the transfer RL concept is helpful for establishing the real-time decision-making policy for autonomous vehicles. [ABSTRACT FROM AUTHOR]

Advanced algorithms can promote the development of energy management strategies (EMSs) as a key technology in hybrid electric vehicles (HEVs). Reinforcement learning (RL) with distributed structure can significantly improve training efficiency in complex environments, and multi-threaded parallel computing provides a reliable algorithm basis for promoting adaptability. Dedicated to trying more efficient deep reinforcement learning (DRL) algorithms, this paper proposed a deep q-network (DQN)-based energy and emission management strategy (E&EMS) at first. Then, two distributed DRL algorithms, namely asynchronous advantage actor-critic (A3C) and distributed proximal policy optimization (DPPO), were adopted to propose EMSs, respectively. Finally, emission optimization was taken into account and then distributed DRL-based E&EMSs were proposed. Regarding dynamic programming (DP) as the optimal benchmark, simulation results show that three DRL-based control strategies can achieve near-optimal fuel economy and outstanding computational efficiency, and compared with DQN, two distributed DRL algorithms have improved the learning efficiency by four times. [ABSTRACT FROM AUTHOR]

Motivated by the concerns on transported fuel consumption and global air pollution, industrial engineers and academic researchers have made many efforts to construct more efficient and environment-friendly vehicles. Hybrid electric vehicles (HEVs) are the representative ones because they can satisfy the power demand by coordinating energy supplements among different energy storage devices. To achieve this goal, energy management approaches are crucial technology, and driving cycles are the critical influence factor. Therefore, this paper aims to summarize driving cycle-driven energy management strategies (EMSs) for HEVs. First, the definition and significance of driving cycles in the energy management field are clarified, and the recent literature in this research domain is reviewed and revisited. In addition, according to the known information of driving cycles, the EMSs are divided into three categories, and the relevant study directions, such as standard driving cycles, long-term driving cycle generation, and short-term driving cycle prediction are illuminated and analyzed. Furthermore, the existing database of driving cycles in highway and urban aspects is displayed and discussed. Finally, this article also elaborates on the future prospects of energy management technologies related to driving cycles. This paper focusing on helping the relevant researchers realize the state-of-the-art of HEVs' energy management field and also recognize its future development direction. • Hybrid electric vehicles (HEVs) as potential on-road vehicles are essentially discussed. • A thorough review of the energy management policy for HEV is conducted. • The learning-based and data-driven methods are analyzed in detail. • Driving cycle databases for further research of HEVs are provided. [ABSTRACT FROM AUTHOR]

Aiming at promoting the intelligent development of control technology for new energy vehicles and showing the outstanding advantages of deep reinforcement learning (DRL), this paper trained a VGG16-based road recognition convolutional neural network (CNN) at first. Lots of high-definition images of five typical roads are collected by the racing game Dust Rally 2.0, including dry asphalt, wet asphalt, snow, dry cobblestone, and wet cobblestone. Then, a time-varying driving environment model was established, involving driving images, road slope, longitudinal speed, and the number of passengers. Finally, a stereoscopic control network suitable for nine-dimensional state space and three-dimensional action space was built, and for parallel hybrid electric vehicles (HEVs) with the P3 structure, a deep q-network (DQN)-based energy management strategy (EMS) achieving multi-objective control was proposed, including the fine-tuning strategy of the motor speed to maintain the optimal slip rate during braking, the engine power control strategy and the continuously variable transmission (CVT) gear ratio control strategy. Simulation results show under the influence of some factors such as tree shade and image compression, the road recognition network has the highest accuracy for snow roads and wet asphalt roads. Three types of control strategies learned simultaneously by the stereoscopic control network not only maintain the near-optimal slip rate in the braking period but also achieve a fuel consumption of 4788.93 g, while dynamic programming (DP)-based EMS gets a fuel consumption of 4295.61 g. Moreover, even DP-based EMS only contains three states and two actions, the time consumed for DP-based EMS and DQN-based EMS to run the speed cycle of 3602s is about 4911s and 10s, respectively. Therefore, the optimization and real-time performance of DRL-based EMS can be guaranteed. • A VGG16-based neural network for road recognition is trained. • A complex and time-varying road model with idealized characteristics is established. • Aiming at achieving integrated control, a stereoscopic control network is modeled. • A DQN-based multi-objective control strategy is proposed. [ABSTRACT FROM AUTHOR]

This research proposes a reinforcement learning-based algorithm and a deep reinforcement learning-based algorithm for energy management of a series hybrid electric tracked vehicle. Firstly, the powertrain model of the series hybrid electric tracked vehicle (SHETV) is constructed, then the corresponding energy management formulation is established. Subsequently, a new variant of reinforcement learning (RL) method Dyna, namely Dyna-H, is developed by combining the heuristic planning step with the Dyna agent and is applied to energy management control for SHETV. Its rapidity and optimality are validated by comparing with DP and conventional Dyna method. Facing the problem of the "curse of dimensionality" in the reinforcement learning method, a novel deep reinforcement learning algorithm deep Q-learning (DQL) is designed for energy management control, which uses a new optimization method (AMSGrad) to update the weights of the neural network. Then the proposed deep reinforcement learning control system is trained and verified by the realistic driving condition with high-precision, and is compared with the benchmark method DP and the traditional DQL method. Results show that the proposed deep reinforcement learning method realizes faster training speed and lower fuel consumption than traditional DQL policy does, and its fuel economy quite approximates to global optimum. Furthermore, the adaptability of the proposed method is confirmed in another driving schedule. • The powertrain model of the series hybrid electric tracked vehicle is constructed. • A novel reinforcement learning-based energy management strategy is proposed. • The rapidity and optimality of the reinforcement learning method are validated. • A new optimization method is applied to update the weights of the neural network. • The proposed deep reinforcement learning method realizes better performance. [ABSTRACT FROM AUTHOR]