Search

With the development of artificial intelligence technology, deep reinforcement learning (DRL) has become a major approach to the design of intelligent vehicle-to-everything (V2X) routing protocols for vehicular ad hoc networks (VANETs). However, if the V2X routing protocol does not consider both real-time traffic conditions and historical vehicle trajectory information, the source vehicle may not transfer its packet to the correct relay vehicles and, finally, to the destination. Thus, this kind of routing protocol fails to guarantee successful packet delivery. Using the greater network flexibility and scalability of the software-defined network (SDN) architecture, this study designs a two-phase integrated DQN and RF Packet Routing Framework (IDRF) that combines the deep Q-learning network (DQN) and random forest (RF) approaches. First, the IDRF offline phase corrects the vehicle's historical trajectory information using the vehicle trajectory continuity algorithm and trains the DQN model. Then, the IDRF real-time phase judges whether vehicles can meet each other and makes a real-time routing decision to select the most appropriate relay vehicle after adding real-time vehicles to the VANET. In this way, the IDRF can obtain the packet transfer path with the shortest end-to-end delay. Compared to two DQN-based approaches, i.e., TDRL-RP and VRDRT, and traditional VANET routing algorithms, the IDRF exhibits significant performance improvements for both sparse and congested periods during intensive simulations of the historical GPS trajectories of 10,357 taxis within Beijing city. Performance improvements in the average packet delivery ratio, end-to-end delay, and overhead ratio of the IDRF over TDRL-RP and VRDRT under different numbers of pairs and transmission ranges are at least 3.56%, 12.73%, and 5.14% and 6.06%, 11.84%, and 7.08%, respectively. [ABSTRACT FROM AUTHOR]

As for the problem that the traditional single depth prediction model has poor strain capacity to the prediction results of time series data when predicting lake eutrophication, this study takes the multi-factor water quality data affecting lake eutrophication as the main research object. A deep reinforcement learning model is proposed, which can realize the mutual conversion of water quality data prediction models at different times, select the optimal prediction strategy of lake eutrophication at the current time according to its own continuous learning, and improve the reinforcement learning algorithm. Firstly, the greedy factor, the fixed parameter of Agent learning training in reinforcement learning, is introduced into an arctangent function and the mean value reward factor is defined. On this basis, three Q estimates are introduced, and the weight parameters are obtained by calculating the realistic value of Q, taking the average value and the minimum value to update the final Q table, so as to get an Improved MIMO-DD-3Q Learning model. The preliminary prediction results of lake eutrophication are obtained, and the errors obtained are used as the secondary input to continue updating the Q table to build the final Improved MIMO-DD-3Q Learning model, so as to achieve the final prediction of water eutrophication. In this study, multi-factor water quality data of Yongding River in Beijing were selected from 0:00 on July 26, 2021 to 0:00 on September 5, 2021. Firstly, data smoothing and principal component analysis were carried out to confirm that there was a certain correlation between all factors in the occurrence of lake eutrophication. Then, the Improved MIMO-DD-3Q Learning prediction model was used for experimental verification. The results show that the Improved MIMO-DD-3Q Learning model has a good effect in the field of lake eutrophication prediction. [ABSTRACT FROM AUTHOR]

With the traffic congestion problem deteriorating, people increasingly choose urban rail transit (URT) to travel. Although URT alleviates traffic congestion, the long-term operation of a large number of trains leads to huge energy consumption. In order to adapt the major social development concept of "Low carbon", a multi-train energy-saving control collaborative optimization method is proposed in this paper. First, the composition of single train operating conditions is determined by the conversion of operating conditions between stations and the force changes under the premise of ensuring safe and on-time train operation. A single-train energy consumption calculation combinatorial optimization model with the dual control objectives of reducing passengers' average waiting time as well as train traction energy consumption is established. The energy saving control strategy of a single train is investigated by ARMA-Radial Basis Function Neural Network (ARMA-RBFNN) and Genetic Algorithm (GA). Next, the queuing theory is introduced to analyze the variation in passenger waiting time for multiple trains at different arrival intervals. A Deep Reinforcement Learning (DRL) algorithm is designed to obtain the correlation among passenger waiting time, arrival interval and train stopping time. The optimization objective is to minimize the multi-train traction energy consumption and the average passenger waiting time while considering conditions such as train operating safety interval, speed limit, multiple operating state and single train energy-saving models, etc. Then, a multi-train cooperative energy-saving control model is proposed based on the Dragonfly Algorithm (DA). Finally, a case study of Beijing Metro Line 4 is conducted to illustrate the effectiveness of the proposed method. The results demonstrate that the total traction energy consumption and passenger waiting time are reduced by 3.1% and 5 s, respectively, compared with the method of independently optimizing the single-train control strategy. The findings can aid in the development of energy-saving strategies and also provide a basis for energy-saving operation control of multiple trains. [ABSTRACT FROM AUTHOR]

Demand responsive transit (DRT) is expected to offer enormous possibilities for fulfilling the ever-growing diversified demand while promoting the urban sustainability. Nevertheless, it seems to remain critical but intractable to make the routing decision in response to the time-dependent travel speeds. Considering the risk of speed uncertainty, deep reinforcement learning (DRL) algorithm is presented to address the time-dependent electric DRT problem in this study. Long short-term memory (LSTM) is integrated into the attention mechanism at the decoding process. To evaluate the sustainability, the reward function can be subdivided into the number of served passengers, regret utility, and carbon emissions. To testify the effectiveness, we made a comparison between the CPLEX solver, NSGA-II, and the proposed algorithm in a realistic transportation network of Beijing. The computational results demonstrate that DRL algorithm with shorter computation time and better solutions is dramatically superior to the other approaches. Therefore, the DRL algorithm provides a more efficient framework for addressing the time-dependent electric DRT problem while improving the sustainability of the environment and society. • Deep reinforcement learning (DRL) algorithm is constructed to address the time-dependent electric DRT problem. • From the perspective of the sustainability, the number of served passengers, regret utility, and carbon emission are presented as the objective function. • To explore the passenger's decision-making psychology, regret theory and indifference threshold are incorporated for the regret utility. • Numerical examples in Beijing network are further investigated to assess environmental and social sustainability. [ABSTRACT FROM AUTHOR]

Traffic prediction is an important management tool for traffic guidance and control and an effective decision-making tool to help travelers plan routes and avoid congested road sections. However, due to the transient and sudden nature of traffic bursts caused by events and data limitations, mainstream methods do not perform well in short-term traffic prediction for special events (SEs). To address this challenge, we propose a traffic burst-sensitive model (TBSM) for short-term traffic prediction. Specifically, we first define a new state unit with the short-term trend and observed state to represent both the burst case and usual case. Second, a state-and-trend unit similarity degree (SD) measurement method and increment-based prediction model are proposed. The key parameter of this model balances the weight of the short-term trend with the observed state. Finally, we use a deep deterministic policy gradient (DDPG) framework containing long short-term memory (LSTM) networks to realize the self-learning and adjustment of weights to ensure the generality and burst sensitivity of the model. The TBSM is implemented in the district of Beijing Workers' Stadium, where SEs occur frequently. The results demonstrate that the proposed model performs significantly better than other traditional machine learning approaches and deep learning approaches for SEs. Our TensorFlow implementation of the TBSM is available at https://github.com/buaajh/TBSM-Traffic-burst-sensitive-model-. [ABSTRACT FROM AUTHOR]