Search

With the development of recent artificial intelligence technology, especially after the great success of AlphaGo, there has been a growing interest in applying reinforcement learning (RL) to solve energy management strategy (EMS) problems for hybrid electric vehicles. However, the issues of current RL algorithms including deployment inefficiency, safety constraint, and simulation-to-real gap make it inapplicable to many industrial EMS tasks. With these in mind and considering the fact that there exists many suboptimal EMS controllers which can generate plentiful amounts of interactive data containing informative behaviors, an offline RL training framework that tries to extract policies with the maximum possible utility out of the available offline data is proposed. Furthermore, with connected vehicle technology standard in many new cars, rather than bringing all the data to the storage and analytics, a scheduled training framework is put forward. This cloud-based approach not only alleviates the computational burden of edge devices, but also more importantly provides a deployment-efficient solution to EMS tasks that have to adapt to changes of driving cycle. To evaluate the effectiveness of the proposed algorithm on real controllers, a hardware-in-the-loop (HIL) test is performed and the superiority of the proposed algorithm in contrast to dynamic programming, behavior cloning, rule-based, and vanilla off-policy RL algorithms is given. [ABSTRACT FROM AUTHOR]

Deep reinforcement learning (DRL) excels at playing a wide variety of simulated games and allows for a generic learning process that does not consider a specific knowledge of the task. However, due to the fact that a large prohibitively number of interactions with the environment are required and that the initial policy behavior is almost random, such an algorithm cannot be trained directly in a real-world environment while satisfying given safety constraints. In this article, a control framework based on DRL that shifts toward training directly in the transient real-world environment is proposed. This research is working on the assumption that some demonstration knowledge that operates under previous controllers and an abstract of the agent environment dynamics are available. By encoding this prior knowledge into a sophisticated learning architecture, a warm-starting DRL algorithm with a safe exploration guarantee can be anticipated. Taking the boost control problem for a variable geometry turbocharger equipped diesel engine as an example, the proposed algorithm improves the initial performance by 74.6% and the learning efficiency by an order of magnitude in contrast to its vanilla counterpart. Compared with other existing DRL-based powertrain control methods, the proposed algorithm can realize the “model-free” concept in the strict sense, making it attractive for future DRL-based powertrain control algorithms to build on. [ABSTRACT FROM AUTHOR]

For the ongoing revolution in developing intelligent and connected vehicles (ICVs), there is a lack of research for powertrain control systems using the latest artificial intelligence and vehicle-to-everything technology that have already been widely adopted in the autonomous driving systems. In this context, recent development of deep reinforcement learning (DRL) and one of the latest computing frameworks are coupled to facilitate an onboard-based intelligent powertrain control. Taking the boost control of a diesel engine equipped with variable geometry turbocharger as an example, the results show that the final control behavior indicated by the cumulated rewards is improved by 50.43% and the learning efficiency is improved by 74.29% for the proposed curiosity-driven DRL algorithm, compared with the same structure DRL algorithm with classic random exploration policy. In addition, unlike most of the DRL-based powertrain optimization algorithms, which have only been applied to single-machine architecture, this work manages the proposed DRL algorithm in parallel and, more importantly, from an edge computing perspective. This, in addition to greatly speeding up the algorithm training, can also realize a good balance of control accuracy and generality depending upon the selected training scenario. Moreover, unlike most of the cloud computing frameworks, which require low network latency, the proposed architecture can achieve a similar final control performance even if good network communication is not allowed. Compared with other existing powertrain control methods, the proposed algorithm is able to approximate a global powertrain control optimization autonomously in a connected manner, making it attractive to current ICVs with advanced automated driving and traditional powertrain control. [ABSTRACT FROM AUTHOR]