Search

Urban traffic management is highly complex, and inefficient control strategies often worsen congestion and increase energy consumption. This paper introduces a collaborative multi-agent reinforcement learning method tailored for sparse control scenarios, IKS-SAC (Improved Knowledge Sharing Soft Actor–Critic), which enhances coordination between traffic signals to optimize traffic flow. IKS-SAC incorporates a communication protocol for knowledge sharing among agents, enabling each agent to access and utilize traffic environment data collected by other agents, effectively addressing the challenge of data processing in asynchronous updates, thereby achieving a comprehensive understanding of the traffic environment within a sparse control framework. Validation of the synthetic data demonstrates that IKS-SAC exhibits superior adaptability and efficiency in managing traffic flow fluctuations and uncertainties, significantly outperforming existing reinforcement learning-based and traditional traffic control methods. The proposed method demonstrates significant advantages in reducing traffic congestion, lowering energy consumption, and mitigating environmental pollution. [ABSTRACT FROM AUTHOR] more...

The scale of electric vehicles (EVs) in microgrids is growing prominently. However, the stochasticity of EV charging behavior poses formidable obstacles to exploring their dispatch potential. To solve this issue, an optimization strategy for EV-integrated microgrids considering peer-to-peer (P2P) transactions has been proposed in this paper. This research strategy contributes to the sustainable development of microgrids under large-scale EV integration. Firstly, a novel cooperative operation framework considering P2P transactions is established, in which the impact factors of EV charging are regarded to simulate its stochasticity and the energy trading process of the EV-integrated microgrid participating in P2P transactions is defined. Secondly, cost models for the EV-integrated microgrid are established. Thirdly, a three-stage optimization strategy is proposed to simplify the solving process. It transforms the scheduling problem into three solvable subproblems and restructures them with Lagrangian relaxation. Finally, case studies demonstrate that the proposed strategy optimizes EV load distribution, reduces the overall operational cost of the EV-integrated microgrid, and enhances the economic efficiency of each microgrid participating in P2P transactions. [ABSTRACT FROM AUTHOR] more...

Due to the dynamic complexities of the multi-unmanned vessel target assignment problem at sea, especially when addressing moving targets, traditional optimization algorithms often fail to quickly find an adequate solution. To overcome this, we have developed a multi-agent reinforcement learning algorithm. This approach involves defining a state space, employing preferential experience replay, and integrating self-attention mechanisms, which are applied to a novel offshore unmanned vessel model designed for dynamic target allocation. We have conducted a thorough analysis of strike positions and times, establishing robust mathematical models. Additionally, we designed several experiments to test the effectiveness of the algorithm. The proposed algorithm improves the quality of the solution by at least 30% in larger scale scenarios compared to the genetic algorithm (GA), and the average solution speed is less than 10% of the GA, demonstrating the feasibility of the algorithm in solving the problem. [ABSTRACT FROM AUTHOR] more...

Copyright of Systems Engineering & Electronics is the property of Journal of Systems Engineering & Electronics Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) more...

A distributed control protocol is proposed in this paper for tracking a leader with a sinusoidal motion path considering an undirected communication graph. A multi-agent system (MAS) is considered in which the dynamics of all agents and the leader are nonlinear and contaminated with noise. Firstly, we assume that the states of the leader and all of the agents are measurable without noise. In this case, graph theory, Lyapunov approach and Lasalle principle are used to design a distributed control protocol for nonlinear MASs to follow the leader with nonlinear dynamics. Next, it is supposed that the leader has a sinusoidal motion and its neighbor followers observe the leader with noise. Therefore, the parameters of measured signal including amplitude, frequency and phase are unknown. In this way, a novel algorithm called Integral Linear Least Square (ILLS) is proposed to estimate unknown parameters of sinusoidal behavior of the leader which is contaminated by noise, accurately. Next, a distributed control algorithm is designed for multiagent systems with Lipschitz nonlinearities under undirected graph to track the estimated states of the leader with noisy sinusoidal motion. Finally, numerical simulations illustrate the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR] more...

A new swarm-based metaheuristic, namely the enriched coati osprey algorithm (ECOA), is proposed in this paper. As its name suggests, ECOA hybridizes two new metaheuristics, the coati optimization algorithm (COA) and the osprey optimization algorithm (OOA). ECOA is constructed by five searches performed sequentially by the swarm members. The first three are directed searches, while the last two are neighborhood searches. All three directed searches are adopted from COA and OOA. Meanwhile, the four-bordered neighborhood search is developed based on a new approach. During the assessment, ECOA was challenged to overcome the set of 23 functions and contended with five new metaheuristics: total interaction algorithm (TIA), golden search optimization (GSO), average and subtraction-based optimization (ASBO), COA, and OOA. The result shows that ECOA outperforms TIA, GSO, ASBO, COA, and OOA in 16, 23, 18, 21, and 21 functions. Meanwhile, the individual search test result shows that the directed searches perform better than the neighborhood searches. Moreover, the directed search toward the best member becomes the most dominant search. [ABSTRACT FROM AUTHOR] more...

To solve the problems of low efficiency of network service caching and computing-networking resource allocation caused by tasks differentiation, highly dynamic network environment, and decentralized computing-networking resource deployment in edge networks, a decentralized service arrangement and computing offloading model for mobile edge computing was investigated and established.Considering the multidimensional resource constraints, e.g., computing power, storage, and bandwidth, with the objective of minimizing task processing latency, the joint optimization of service caching and computing-networking resource allocation was abstracted as a partially observable Markov decision process.Considering the temporal dependency of service request and its coupling relationship with service caching, a long short-term memory network was introduced to capture time-related network state information.Then, based on recurrent multi-agent deep reinforcement learning, a distributed service arrangement and resource allocation algorithm was proposed to autonomously decide service caching and computing-networking resource allocation strategies.Simulation results demonstrate that significant performance improvements in terms of cache hit rate and task processing latency achieved by the proposed algorithm. more...

Copyright of Journal on Communication / Tongxin Xuebao is the property of Journal on Communications Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) more...

As one of the important branches in the field of machine learning and artificial intelligence, fully cooperative multi- agent deep reinforcement learning effectively combines the expression and decision-making ability of deep reinforcement learning with the distributed cooperation ability of multi-agent system in a general way, which provides an endto- end solution to the model-free sequential decision-making problem in fully cooperative multi-agent system. Firstly, the basic principles of deep reinforcement learning are described, and the development of single agent deep reinforcement learning is summarized from three main directions: value function based, policy gradient based and actor-critic based. Secondly, the main challenges and training framework of multi-agent deep reinforcement learning are analyzed. Then, according to the different ways of realizing the maximum team joint reward, the fully cooperative multi-agent deep reinforcement learning is divided into four categories: independent learning, communication learning, collaborative learning and reward function shaping. Finally, from the perspective of solving practical problems, the future development direction of fully cooperative multi-agent deep reinforcement learning algorithm is prospected. [ABSTRACT FROM AUTHOR] more...

Traffic lights play a vital role in the operation of the traffic network. However, with the rapid development of traffic, roads are becoming more and more complex, and vehicles are becoming more and more numerous, which leads to the increasing pressure of traffic lights scheduling, but the regulation ability is becoming weaker and weaker. In order to solve this problem, this paper established the convergence trace source (CTS) scheme. This scheme used the traffic light, the main object of traffic diversion, as an agent for reinforcement learning to optimize its ability to control traffic diversion. It comprehensively analyzed the congestion situation of the road network by constructing the congestion chain and congestion ring, and used the traffic light phase and its timing data to achieve the comprehensive judgment of the object state of the traffic light agent. This scheme designed the traffic light queue length algorithm, and used the digitization of congestion as an agent reward to evaluate the optimization effect. This paper used the SUMO simulation environment for experiments, designed and compared the average queue length at the intersection of the traffic optimization index. Finally, the average queue length at the intersection of this scheme is increased by 40% compared with the original data. [ABSTRACT FROM AUTHOR] more...

Abstract The option of decarbonizing urban freight transport using battery electric vehicle (BEV) seems promising. However, there is currently a strong debate whether fuel cell electric vehicle (FCEV) might be the better solution. The question arises as to how a fleet of FCEV influences the operating cost, the greenhouse gas (GHG) emissions and primary energy demand in comparison to BEVs and to Internal Combustion Engine Vehicle (ICEV). To investigate this, we simulate the urban food retailing as a representative share of urban freight transport using a multi-agent transport simulation software. Synthetic routes as well as fleet size and composition are determined by solving a vehicle routing problem. We compute the operating costs using a total cost of ownership analysis and the use phase emissions as well as primary energy demand using the well to wheel approach. While a change to BEV results in 17–23% higher costs compared to ICEV, using FCEVs leads to 22–57% higher costs. Assuming today’s electricity mix, we show a GHG emission reduction of 25% compared to the ICEV base case when using BEV. Current hydrogen production leads to a GHG reduction of 33% when using FCEV which however cannot be scaled to larger fleets. Using current electricity in electrolysis will increase GHG emission by 60% compared to the base case. Assuming 100% renewable electricity for charging and hydrogen production, the reduction from FCEVs rises to 73% and from BEV to 92%. The primary energy requirement for BEV is in all cases lower and for higher compared to the base case. We conclude that while FCEV have a slightly higher GHG savings potential with current hydrogen, BEV are the favored technology for urban freight transport from an economic and ecological point of view, considering the increasing shares of renewable energies in the grid mix. more...

Providing security for the customers in the e-commerce system is an essential issue. Providing security for each single online customer at the same time is considered a time consuming process. For a huge websites such task may cause several problems including response delay, losing the customer orders and system deadlock or crash, in which reduce system performance. This paper aims to provide a new prototype structure of multi agent system that solve the problem of providing security and avoid the problems that may reduce system performance. This is done by creating a software agent which is settled into the customer device to be responsible of purchase and encryption process without the customer interfering. The proposed agent avoids the problem of deadlock (i.e., break down) and the loss of requests which provides the required protection for information transmitted among all entities. Experimental results showed that employing software agent to manage purchase and encryption tasks improve system performance by 10% and increase system response time by 30.5%. [ABSTRACT FROM AUTHOR] more...

Copyright of Experimental Technology & Management is the property of Experimental Technology & Management Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) more...

The popularity of multi-agent deep reinforcement learning (MADRL) is growing rapidly with the demand for large-scale real-world tasks that require swarm intelligence, and many studies have improved MADRL from the perspective of network structures or reinforcement learning methods. However, the application of MADRL in the real world is hampered by the low sample efficiency of the models and the high cost to collect data. To improve the practicability, an extension to the current training paradigm of MADRL that improves the sample efficiency is imperative. To this end, this paper proposes PEDMA, a flexible plugin unit for MADRL. It consists of three techniques: (i)Parallel Environments (PE), to accelerate the data acquisition; (ii)Experience Augmentation (EA), a novel data augmentation method that utilizes the permutation invariance property of the multi-agent system to reduce the cost of acquiring data; and (iii)Delayed Updated Policies (DUP), to improve the data utilization efficiency of the MADRL model. The proposed EA method could improve the performance, data efficiency, and convergence speed of MADRL models, which is theoretically and empirically demonstrated. Experiments on three multi-agent benchmark tasks show that the MAAC model trained with PEDMA outperforms the baselines and state-of-the-art algorithms, and ablation studies show the contribution and necessity of each component in PEDMA. [ABSTRACT FROM AUTHOR] more...

The option of decarbonizing urban freight transport using battery electric vehicle (BEV) seems promising. However, there is currently a strong debate whether fuel cell electric vehicle (FCEV) might be the better solution. The question arises as to how a fleet of FCEV influences the operating cost, the greenhouse gas (GHG) emissions and primary energy demand in comparison to BEVs and to Internal Combustion Engine Vehicle (ICEV). To investigate this, we simulate the urban food retailing as a representative share of urban freight transport using a multi-agent transport simulation software. Synthetic routes as well as fleet size and composition are determined by solving a vehicle routing problem. We compute the operating costs using a total cost of ownership analysis and the use phase emissions as well as primary energy demand using the well to wheel approach. While a change to BEV results in 17–23% higher costs compared to ICEV, using FCEVs leads to 22–57% higher costs. Assuming today's electricity mix, we show a GHG emission reduction of 25% compared to the ICEV base case when using BEV. Current hydrogen production leads to a GHG reduction of 33% when using FCEV which however cannot be scaled to larger fleets. Using current electricity in electrolysis will increase GHG emission by 60% compared to the base case. Assuming 100% renewable electricity for charging and hydrogen production, the reduction from FCEVs rises to 73% and from BEV to 92%. The primary energy requirement for BEV is in all cases lower and for higher compared to the base case. We conclude that while FCEV have a slightly higher GHG savings potential with current hydrogen, BEV are the favored technology for urban freight transport from an economic and ecological point of view, considering the increasing shares of renewable energies in the grid mix. [ABSTRACT FROM AUTHOR] more...

The highly dynamic nature of stock markets has motivated researchers to propose various supervised learning models to assist investors to optimize financial performance. Machine learning models have been used to predict price trends, and approaches have been proposed for portfolio management. However, these studies focus on only one kind of financial issue, and the methods proposed exhibit poor generalizability. We address these problems with a multi-agent portfolio adaptive trading framework based on reinforcement learning to create an automated trading system with the best trading strategy that can be achieved by long-short situation judgment and adaptive capital allocation. We use the TD3 algorithm in the multi-agent algorithm to mitigate the overestimation and overfitting exhibited by traditional value functions and improve training stability. Experimental results show that the proposed framework outperforms single-agent reinforcement learning algorithms while achieving more stable returns. • This study seeks to optimize trading strategy and portfolio management issues using RL. • The multi-agent architecture allows agents to explore chaotic environments aside from their own capital assets. • The adaptive TPM autonomously rates assets, optimizing Sortino ratio rewards through dynamic trading weights. • Experimental results show that the proposed framework outperforms single-agent reinforcement learning algorithms. [ABSTRACT FROM AUTHOR] more...

Although multi-agent reinforcement learning (MARL) is a promising method for learning a collaborative action policy, enabling each agent to accomplish specified tasks, MARL has a problem of exponentially increasing state-action space. This state-action space can be dramatically reduced by assuming sparse interaction. We previously proposed three methods (greedily selecting actions, switching between Q-value update equations on the basis of the state of each agent in the next step, and their combination) for improving the performance of coordinating Q-learning (CQ-learning), a typical method for multi-agent reinforcement learning with sparse interaction. We have now modified the learning algorithm used in a combination of these two methods to enable it to cope with interference among more than two agents. Evaluation of this enhanced method using two additional maze games from three perspectives (the number of steps to a goal, the number of augmented states, and the computational cost) demonstrated that the modified algorithm improves the performance of CQ-learning. more...

In order to quickly and accurately locate the fault location of the distribution network and increase the stability of the distribution network, a fault recovery method based on multi-objective optimization algorithm is proposed. The optimization of the power distribution network fault system based on multiagent technology realizes fast recovery of multi-objective fault, solve the problem of network learning and parameter adjustment in the later stage of particle swarm optimization algorithm falling into the local extreme value dilemma, and realize the multi-dimensional nonlinear optimization of the main grid and the auxiliary grid. The system proposed in this study takes power distribution network as the goal, applies fuzzy probability algorithm, simplifies the calculation process, avoids local extreme value, and finally realizes the energy balance between each power grid. Simulation results show that the Multi-Agent Technology enjoys priority in restoring important load, shortening the recovery time of power grid balance, and reducing the overall line loss rate of power grid. Therefore, the power grid fault self-healing system can improve the safety and stability of the important power grid, and reduce the economic loss rate of the whole power grid. [ABSTRACT FROM AUTHOR] more...

This paper presents a framework of load shedding experiment for a DC Microgrid using Multi-Agent System (MAS). The microgrid uses solar panels as source of energy to serve a community without access to electricity. The generated framework includes modelling of solar panels, battery storage and loads for effective control and better operation. The loads are classified as critical and non-critical loads. The agents are designed in a decentralized manner which include solar agent, storage agent and load agent. The load shedding experiment of the framework is mapped with the manual operation done at Kisiju village, Pwani, Tanzania. The results of the experiment focus on using accurate solar and PV panels which provide: (i) the multi agent system that runs in the DC microgrid, (ii) the controlling and monitoring of power to be used for critical and non-critical loads and (ii) the management power in the production process through selling extra power from an individual load to the storage. more...

Copyright of Przegląd Elektrotechniczny is the property of Przeglad Elektrotechniczny and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) more...

In the large cities, it remains difficult and expensive to create more parking spaces for vehicles since they have almost reached their full occupancy. The lack of available parking places leads to the problem of traffic congestion and consequently to pollution, since drivers will spend a lot of time looking for a vacant parking place. Hence the need to establish a parking management system able to improve the exploitation of the existent parking places at a city level. This paper proposes a new multi agent system for parking vacancies detection based on vision techniques. We explore a network of interconnected video surveillance cameras and parking stations to provide an intelligent service to the drivers in order to facilitate their task and improve the exploitation of the parking resources. The proposed system introduces a new approach for parking spaces modeling and elaborates an adaptive approach for vacancies estimation in order to supply the driver with reliable information about the vacant parking spaces in the city according to the size of his vehicle. [ABSTRACT FROM AUTHOR] more...