Search

Autonomous vehicles (AVs) are expected to operate on mobility-on-demand (MoD) platforms because AV technology enables flexible self-relocation and system-optimal coordination. Unlike the existing studies, which focus on MoD with pure AV fleet or conventional vehicles (CVs) fleet, we aim to optimize the real-time fleet management of an MoD system with a mixed autonomy of CVs and AVs. We consider a realistic case that heterogeneous boundedly rational drivers may determine and learn their relocation strategies to improve their own compensation. In contrast, AVs are fully compliant with the platform's operational decisions. To achieve a high level of service provided by a mixed fleet, we propose that the platform prioritizes human drivers in the matching decisions when on-demand requests arrive and dynamically determines the AV relocation tasks and the optimal commission fee to influence drivers' behavior. However, it is challenging to make efficient real-time fleet management decisions when spatiotemporal uncertainty in demand and complex interactions among human drivers and operators are anticipated and considered in the operator's decision making. To tackle the challenges, we develop a two-sided multiagent deep reinforcement learning (DRL) approach in which the operator acts as a supervisor agent on one side and makes centralized decisions on the mixed fleet, and each CV driver acts as an individual agent on the other side and learns to make decentralized decisions noncooperatively. We establish a two-sided multiagent advantage actor-critic algorithm to simultaneously train different agents on the two sides. For the first time, a scalable algorithm is developed here for mixed fleet management. Furthermore, we formulate a two-head policy network to enable the supervisor agent to efficiently make multitask decisions based on one policy network, which greatly reduces the computational time. The two-sided multiagent DRL approach is demonstrated using a case study in New York City using real taxi trip data. Results show that our algorithm can make high-quality decisions quickly and outperform benchmark policies. The efficiency of the two-head policy network is demonstrated by comparing it with the case using two separate policy networks. Our fleet management strategy makes both the platform and the drivers better off, especially in scenarios with high demand volume. History: This paper has been accepted for the Transportation Science Special Issue on Emerging Topics in Transportation Science and Logistics. Funding: This work was supported by the Singapore Ministry of Education Academic Research [Grant MOE2019-T2-2-165] and the Singapore Ministry of Education [Grant R-266-000-135-114]. [ABSTRACT FROM AUTHOR]

We consider the problem of an operator controlling a fleet of electric vehicles for use in a ride-hailing service. The operator, seeking to maximize profit, must assign vehicles to requests as they arise as well as recharge and reposition vehicles in anticipation of future requests. To solve this problem, we employ deep reinforcement learning, developing policies whose decision making uses Q -value approximations learned by deep neural networks. We compare these policies against a reoptimization-based policy and against dual bounds on the value of an optimal policy, including the value of an optimal policy with perfect information, which we establish using a Benders-based decomposition. We assess performance on instances derived from real data for the island of Manhattan in New York City. We find that, across instances of varying size, our best policy trained with deep reinforcement learning outperforms the reoptimization approach. We also provide evidence that this policy may be effectively scaled and deployed on larger instances without retraining. [ABSTRACT FROM AUTHOR]

Greenhouses are key to ensuring food security and realizing a sustainable future for agriculture. However, to ensure crop growth efficiency, greenhouses consume a significant amount of energy, primarily through climate control and artificial lighting systems. Owing to this high energy consumption, a network of greenhouses exhibits immense potential to participate in demand response programs for power grid stability. In this work, a multi-agent deep reinforcement learning (MADRL) control framework utilizing an actor-critic algorithm with a shared attention mechanism is proposed for energy management in networked greenhouses. A network of renewable energy integrated greenhouses is constructed to interact with the power grid, when necessary, to address the fluctuations associated with renewable energy generation and dynamic electricity prices. The viability and scalability of this multi-agent approach is demonstrated by evaluating its capabilities for a network of five greenhouses of varying capacities. The proposed MADRL-based control approach for demand-side energy management in networked greenhouses demonstrates efficiency in maintaining indoor climate in all greenhouses while ensuring a 28% reduction in net load demand as compared to well-known algorithms. • Operation of networked grid-interactive greenhouses is formulated as a Markov game. • Battery energy storage and photovoltaic power generation modeled in each greenhouse. • Multi-agent deep reinforcement learning technique for demand response is developed. • Attention mechanism is leveraged to promote coordination among greenhouses. • Net load reduction and scalability is demonstrated with case study in New York City. [ABSTRACT FROM AUTHOR]

Drones can be used for tasks such as data collection and logistics in civil engineering. Current research on autonomous drones mainly focuses on planning a safe path and avoiding obstacles in a static environment. However, navigating a drone in complex environments like urban areas involves many dynamic constraints, such as building layout, winds, and signal coverage, which are interdependent. The wind factor is the most important among these environmental factors, which may cause a drone to lose control or even crash. This paper presents a multi-objective navigation reinforcement learning algorithm (MONRL) for the drone to navigate and avoid obstacles in an unknown environment when dynamic wind zones present, with only imagery data about the building layouts. Based on a deep reinforcement learning and memory architecture, the drone develops policies to prioritize navigation decisions, optimizing the path while minimizing negative impact of winds with only sparse sensor data, in our case, camera inputs. By leveraging the advantages of the proposed method in estimating the environmental factors from previous trials, no aerodynamic force sensors are needed for the drone to develop effective strategies to navigate to target while counteracting to milder winds and dodging aways from stronger winds. The proposed method was tested in a virtual environment, and a real model of New York City. The method is expected to contribute to new automation algorithms for urban aerial logistics and future automated civil infrastructure inspection. • A multi-objective reinforcement learning method was proposed for autonomous drones to avoid dynamic winds while navigating. • By leveraging a memory RL architecture, no aerodynamic force sensors are needed to sense and react to winds. • A framework for representing complex aerodynamic forces with mathematical formulation was proposed to supportRL trials. • Experiments performed in a VR simulator with data from New York City to confirm the efficacy of the proposed method. [ABSTRACT FROM AUTHOR]

• Deep stochastic reinforcement learning based approach to address this issue of epistemic uncertainty in a midsize fuel cell hybrid electric vehicle. • The performance of the proposed approach is benchmarked against Double Deep Q-Network (DDQN), Power Follower Controller (PFC) and Fuzzy Logic Controller (FLC). • Using New York City cycle as a validation drive cycle, the approach improves fuel economy by 7.68%, 13.53%, and 10% compared to DDQN, PFC, and FLC, respectively. • The deep REINFORCE approach improves fuel economy by 5.31 %,9.78 %, and 9.93 % compared to DDQN, PFC, and FLC, respectively under another validation cycle, Amman cycle. • The proposed method has 38% less training time when compared to the DDQN approach. Fuel cell hybrid electric vehicles offer a promising solution for sustainable and environment friendly transportation, but they necessitate efficient energy management strategies (EMSs) to optimize their fuel economy. However, designing an optimal leaning-based EMS becomes challenging in the presence of limited training data. This paper presents a deep stochastic reinforcement learning based approach to address this issue of epistemic uncertainty in a midsize fuel cell hybrid electric vehicle. The approach introduces a deep REINFORCE framework with a deep neural network baseline and entropy regularization to develop a stochastic policy for EMS. The performance of the proposed approach is benchmarked against three EMSs: i) a state-of- art deep deterministic reinforcement learning technique called Double Deep Q-Network (DDQN), Power Follower Controller (PFC) and Fuzzy Logic Controller (FLC). Using New York City cycle as a validation drive cycle, the deep REINFORCE approach improves fuel economy by 7.68%, 13.53%, and 10% compared to DDQN, PFC, and FLC, respectively. The deep REINFORCE approach improves fuel economy by 5.31 %,9.78 %, and 9.93 % compared to DDQN, PFC, and FLC, respectively under another validation cycle, Amman cycle. Moreover, the training results show that the proposed algorithm reduces training time by 38% compared to the DDQN approach. The proposed deep REINFORCE-based EMS shows superiority not only in terms of fuel economy, but also in terms of dealing with epistemic uncertainty. [ABSTRACT FROM AUTHOR]

Vehicle mobility optimization in urban areas is a long-standing problem in smart city and spatial data analysis. Given the complex urban scenario and unpredictable social events, our work focuses on developing a mobile sequential recommendation system to maximize the profitability of vehicle service providers (e.g., taxi drivers). In particular, we treat the dynamic route optimization problem as a long-term sequential decision-making task. A reinforcement-learning framework is proposed to tackle this problem, by integrating a self-check mechanism and a deep neural network for customer pick-up point monitoring. To account for unexpected situations (e.g., the COVID-19 outbreak), our method is designed to be capable of handling related environment changes with a self-adaptive parameter determination mechanism. Based on the yellow taxi data in New York City and vicinity before and after the COVID-19 outbreak, we have conducted comprehensive experiments to evaluate the effectiveness of our method. The results show consistently excellent performance, from hourly to weekly measures, to support the superiority of our method over the state-of-the-art methods (i.e., with more than 98% improvement in terms of the profitability for taxi drivers). [ABSTRACT FROM AUTHOR]

Recent studies show that 85% of women have changed their traveled routes to avoid harassment and assault. Despite this, current mapping tools do not empower users with information to take charge of their personal safety. We propose SafeRoute, a novel solution to the problem of navigating cities and avoiding street harassment and crime. Unlike other street navigation applications, SafeRoute introduces a new type of path generation via deep reinforcement learning. This enables us to successfully optimize for multi-criteria path-finding and incorporate representation learning within our framework. Our agent learns to pick favorable streets to create a safe and short path with a reward function that incorporates safety and efficiency. Given access to recent crime reports in many urban cities, we train our model for experiments in Boston, New York, and San Francisco. We test our model on areas of these cities, specifically the populated downtown regions with high foot traffic. We evaluate SafeRoute and successfully improve over state-of-the-art methods by up to 17% in local average distance from crimes while decreasing path length by up to 7%. [ABSTRACT FROM AUTHOR]