Search

During a deep excavation project, monitoring the structural health of the adjacent buildings is crucial to ensure safety. Therefore, this study proposes a novel probabilistic deep reinforcement learning (PDRL) framework to optimize the monitoring plan to minimize the cost and excavation‐induced risk. First, a Bayesian‐bi‐directional general regression neural network is built as a probabilistic model to describe the relationship between the ground settlement of the foundation pit and the safety state of the adjacent building, along with the actions in a dynamic manner. Subsequently, a double deep Q‐network method, which can capture the realistic features of the excavation management problem, is trained to form a closed decision loop for continuous learning of monitoring strategies. Finally, the proposed PDRL approach is applied to a real‐world deep excavation case in No. 14 Shanghai Metro. This approach can estimate the time‐variant probability of damage occurrence and maintenance actions and update the state of the adjacent building. According to the strategy proposed via PDRL, monitoring of the adjacent buildings begins in the middle stage rather than on the first day of the excavation project if there is full confidence in the quality of the monitoring data. When the uncertainty level of data rises, the starting day might shift to an earlier date. It is worth noting that the proposed PDRL method is adequately robust to address the uncertainties embedded in the environment and model, thus contributing to optimizing the monitoring plan for achieving cost‐effectiveness and risk mitigation. [ABSTRACT FROM AUTHOR]

With the rapid development of global trade, ports and terminals are playing an increasingly important role, and automatic guided vehicles (AGVs) have been used as the main carriers performing the loading/unloading operations in automated container terminals. In this paper, we investigate a multi-AGV dynamic scheduling problem to improve the terminal operational efficiency, considering the sophisticated complexity and uncertainty involved in the port terminal operation. We propose to model the dynamic scheduling of AGVs as a Markov decision process (MDP) with mixed decision rules. Then, we develop a novel adaptive learning algorithm based on a deep Q-network (DQN) to generate the optimal policy. The proposed algorithm is trained based on data obtained from interactions with a simulation environment that reflects the real-world operation of an automated in Shanghai, China. The simulation studies show that, compared with conventional scheduling methods using a heuristic algorithm, i.e., genetic algorithm (GA) and rule-based scheduling, terminal the proposed approach performs better in terms of effectiveness and efficiency. [ABSTRACT FROM AUTHOR]

Currently, self-driving, emerging as a key automatic application, has brought a huge potential for the provision of in-vehicle services (e.g., automatic path planning) to mitigate urban traffic congestion and enhance travel safety. To provide high-quality vehicular services with stringent delay constraints, edge computing (EC) enables resource-hungry self-driving vehicles (SDVs) to offload computation-intensive tasks to the edge servers (ESs). In addition, caching highly reusable contents decreases the redundant transmission time and improves the quality of services (QoS) of SDVs, which is envisioned as a supplement to the computation offloading. However, the high mobility and time-varying requests of SDVs make it challenging to provide reliable offloading decisions while guaranteeing the resource utilization of content caching. To this end, in this paper we propose a collaborative computation offloading and content caching method, named CoPace, by leveraging deep reinforcement learning (DRL) in EC for self-driving system. Specifically, we first introduce OSTP to predict the future time-varying content popularity, taking into account the temporal-spatial attributes of requests. Moreover, a DRL-based algorithm is developed to jointly optimize the offloading and caching decisions, as well as the resource allocation (i.e., computing and communication resources) strategies. Extensive experiments with real-world datasets in Shanghai, China, are conducted to evaluate the performance, which demonstrates that CoPace is both effective and well-performed. [ABSTRACT FROM AUTHOR]

This study proposes a novel eco-driving control strategy for connected and automated hybrid electric vehicles, which utilizes deep reinforcement learning (DRL) to optimize various aspects of driving performance, including fuel economy, ride comfort, and travel efficiency, in complex urban traffic scenarios. The proposed strategy incorporates a driving safety model that predicts potential risk associated with the DRL agent's planned speed, thus ensuring the safety of the DRL based eco-driving strategy. Additionally, we propose a multi-objective composite reward function design scheme that considers various constraints caused by traffic elements, such as traffic lights, preceding vehicles, road curvature, and speed limit. This design scheme enables the proposed strategy to effectively adapt to diverse driving challenges in complex urban traffic scenarios. To evaluate the proposed strategy, we develop an urban traffic simulation model based on real-world road and traffic data from Shanghai, China. This model is used as the test scenario and can reflect real urban traffic conditions. The simulation results demonstrate the capability of the proposed strategy to safely and efficiently control vehicles to complete driving tasks in complex urban scenarios. Moreover, the proposed strategy excels in simultaneously optimizing the driving comfort and fuel consumption of the controlled vehicle. • A novel eco-driving strategy using DRL to optimize driving performance. • The strategy is designed for urban traffic scenarios with multiple traffic factors. • Proposed strategy co-optimizes economy, comfort, travel efficiency, and safety. • Driving safety model predicts risk associated with DRL agent's planned speed. • Composite reward function promotes convergence and satisfies complex constraints. [ABSTRACT FROM AUTHOR]

• We propose a speed control framework for AVs on rough pavements in a vehicle-to-infrastructure communication system. • The concept of 'maximum comfortable speed' is proposed to represent the vertical ride comfort of oncoming roads. • A deep reinforcement learning algorithm is designed to learn comfortable and energy-efficient speed control strategies. • The results indicate that the proposed approach is advantageous and applicable in real-time speed controls on rough pavements. Rough pavements cause ride discomfort and energy inefficiency for road vehicles. Existing methods to address these problems are time-consuming and not adaptive to changing driving conditions on rough pavements. With the development of sensor and communication technologies, crowdsourced road and dynamic traffic information become available for enhancing driving performance, particularly addressing the discomfort and inefficiency issues by controlling driving speeds. This study proposes a speed control framework on rough pavements, envisioning the operation of autonomous vehicles based on the crowdsourced data. We suggest the concept of 'maximum comfortable speed' for representing the vertical ride comfort of oncoming roads. A deep reinforcement learning (DRL) algorithm is designed to learn comfortable and energy-efficient speed control strategies. The DRL-based speed control model is trained using real-world rough pavement data in Shanghai, China. The experimental results show that the vertical ride comfort, energy efficiency, and computation efficiency increase by 8.22%, 24.37%, and 94.38%, respectively, compared to an optimization-based speed control model. The results indicate that the proposed framework is effective for real-time speed controls of autonomous vehicles on rough pavements. [ABSTRACT FROM AUTHOR]