Search

Cloud computing is widely applied in modern industrial areas due to its technological advancement, cost reduction, and applicability. Packets (tasks) belonging to different applications (agents) compete to share the common cloud resource through a series of edge nodes (processors) in pursuit of fast transmission. This paper abstracts the cloud computing system as a multi-agent flow-shop scheduling (MAFS) problem. The objective is to minimise the total completion time of several agents with the restriction that the maximum lateness cannot exceed a given bound. Given the complexity of the considered problem, a branch and bound algorithm combined with several pruning rules and lower bounds is proposed to obtain optimal solutions. Furthermore, the considered problem is generalised to a bi-scenario version, and a bi-population cooperative co-evolutionary (BCCE) algorithm is proposed to solve it. A reinforcement learning-based method is presented to generate the initial population. Several problem-specific intensification strategies are constructed to explore promising solutions. Comprehensive experiments verified the effectiveness of the proposed algorithms. The industrial data from the China Earthquake Network Centre further confirmed the superiority of the BCCE algorithm. Overall, the MAFS model and the proposed algorithms effectively enhance the user experience and reasonably guarantee revenue. [ABSTRACT FROM AUTHOR]

Copyright of Modern Educational Technology is the property of Editorial Board of Modern Educational Technology, Tsinghua University and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

To address the unprecedented challenges of construction pressurized by the global climate crisis, housing shortage, and growing shortage of skilled labor, this research presents a radical shift in the construction lifecycle of buildings, from linear processes that produce static continuous buildings to interrelational processes linking adaptative eco-systems of collaborative robots and reconfigurable building parts. Inspired by natural builders, the interdisciplinary field of collective robotic construction (CRC) offers the potential for scalable, adaptive, and resilient construction with simple robots. We establish a design framework for autonomous collaborative robotic construction (ACRC) through modular robotic material eco-systems (MRMES) trained with deep multi-agent reinforcement learning (DMARL). This involves the integration of three core aspects: (1) modular robotic material eco-systems (2) cyber-physical simulation and control with bidirectional feedback (3) adaptive intelligence through deep multi-agent reinforcement learning. The framework is implemented through three comparable case studies for collaborative modular robotic assembly of reconfigurable building parts. [ABSTRACT FROM AUTHOR]

The deterioration of bridge networks poses a major threat to the availability and function of transportation systems and ultimately affects social development. Deep reinforcement learning is expected to provide intelligent decision support for bridge network maintenance. However, existing studies have neglected to explicitly consider the impact of maintenance behavior on the cost-effectiveness of bridge networks. The complex traffic environment and the interconnection of bridge networks also pose unique challenges in balancing maintenance costs and benefits. It is necessary to explore how to use the specific traffic data of each bridge in the bridge network to effectively balance cost-effectiveness and rationalize maintenance decisions. Aiming at the maintenance requirements of the bridge network, a multi-agent ranking proximal policy optimization framework is proposed. The performance of the proposed framework is rigorously evaluated using a real bridge network example. The results show that the maintenance policy based on the proposed framework can maximize the cost-effectiveness of the bridge network in its life cycle, effectively reduce the excessive risk cost and achieve a harmonious balance between different costs. In addition, the proposed framework is superior to the traditional maintenance policy and provides higher performance and efficiency. [ABSTRACT FROM AUTHOR]

No matter whether with constraint or without constraint, most of the research about distributed optimization is studied for the kind of quadratic performance criteria that does not have an integrator; these optimization problems only concern the state value at the time of the final state, not the whole process of the system change. For this problem, this paper discusses second-order multi-agent systems with a discrete-time dynamic and a continuous-time dynamic, respectively, for distributed optimization control problems, and proposes sufficient conditions to ensure the quadratic performance criteria with an integrator is positive. Specifically, under sufficient conditions, we describe the multi-agent systems that are considered in this paper to be connected topology; all the agents can obtain the information from their neighbors. In addition, the structure of our controller only relies on the Laplace matrix of the system's topology, and the reaction coefficients in the controller are the parameters in the performance criteria. Finally, the analysis of convergence is given and verified by numerical examples and simulations. [ABSTRACT FROM AUTHOR]

The evacuation process after the earthquake is affected by different shelters, which can easily lead to disorderly evacuation and prolong the overall evacuation time. Therefore, the scientific planning and design of the service space of shelters in urban disaster prevention areas is necessary to ensure efficient and rational evacuation. In this paper, Set Pair Analysis-Variable Fuzzy Recognition Model (SPA-VFRM) was used to analyze the risk of evacuation disorder in urban disaster prevention areas. Besides, the method for dividing the service scope of medium- and long-term shelters (MLS) based on multi-agent simulation was proposed to improve the evacuation efficiency, which considered the characteristics of evacuation space and the people composition in different types of areas. Taking Xuchang City in China as an example, the multi-agent dynamic simulation experiment was carried out based on the evacuation demand after the earthquake. The results showed that the disorderly evacuation harmed the overall evacuation time and people refuge distribution in the area. When the service radius of MLS in industrial, residential, and comprehensive areas were 2.0 km, 1.0 km, and 1.5 km, respectively, the people evacuation in the area could be completed within 1 h, which met the requirements of the long-distance evacuation (LDE) time in the specification. The research results can provide a basis for the quantitative optimization of the service space for MLS in different types of urban areas, and enrich and deepen the design theory of urban disaster prevention shelters. [ABSTRACT FROM AUTHOR]

Keyphrase generation is an essential tool in the field of natural language processing for information retrieval, document summarization, and text recommendation applications, extracting succinct and representative phrases from the text document. Traditional keyphrase extraction methods applied the supervised or unsupervised learning fail to capture the sequential keyphrase generation in a dynamic environment. The keyphrase generation approaches lack focus on explicitly discriminating the present and absent keyphrases, leading to the inadequate generation of semantically rich absent keyphrases. Hence, this work utilizes the potential benefits of reinforcement learning with the design of a distinguished reward function for present and absent keyphrases for sequential decision-making in the keyphrase generation. Thus, this work presents a novel keyphrase generation system, MADeGen, utilizing Multi-Agent Deep Reinforcement Learning (MADRL). In particular, a multi-agent reinforcement system collaboratively enables the generation of representative and coherent keyphrases by the evaluation metric-aware cooperative reward function analysis and adaptively training the agents. The proposed MADeGen incorporates two major phases, such as multi-agent modelling and actor critic-based policy optimization towards accurate keyphrase generation. In the first phase, the proposed approach designs two learning agents, including the extraction agent and generation agent, with the incorporation of a pre-trained language model. In the multi-agent system, the generation agent is the finetuned version of the extraction agent with the integration of the Wikipedia source. Secondly, the evaluation-aware adaptive reward function is designed to evaluate each agent's generated keyphrases with reference to ground-truth keyphrases. In subsequence, the cooperative reward analysis triggers the actor critic-based policy optimization for the generation agent in the multi-agent system to precisely generate the semantically relevant keyphrases with the assistance of an external web source. Experimental results on several benchmark datasets, such as Inspec, PubMed, and wiki20, illustrate the effectiveness of the proposed MADeGen compared to the existing keyphrase extraction models, yielding state-of-the-art performance in keyphrase extraction tasks. The proposed MADeGen proves its higher performance in the present as well as absent keyphrase extraction as 0.367 and 0.438 F1-score, respectively, while testing on the Inspec dataset. [ABSTRACT FROM AUTHOR]

Copyright of Electric Power is the property of Electric Power Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This paper investigates a Local Strategy-Driven Multi-Agent Deep Deterministic Policy Gradient (LSD-MADDPG) method for demand-side energy management systems (EMS) in smart communities. LSD-MADDPG modifies the conventional MADDPG framework by limiting data sharing during centralized training to only discretized strategic information. During execution, it relies solely on local information, eliminating post-training data exchange. This approach addresses critical challenges commonly faced by EMS solutions serving dynamic, increasing-scale communities, such as communication delays, single-point failures, scalability, and nonstationary environments. By leveraging and sharing only strategic information among agents, LSD-MADDPG optimizes decision-making while enhancing training efficiency and safeguarding data privacy—a critical concern in the community EMS. The proposed LSD-MADDPG has proven to be capable of reducing energy costs and flattening the community demand curve by coordinating indoor temperature control and electric vehicle charging schedules across multiple buildings. Comparative case studies reveal that LSD-MADDPG excels in both cooperative and competitive settings by ensuring fair alignment between individual buildings' energy management actions and community-wide goals, highlighting its potential for advancing future smart community energy management. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicles (UAVs) assisted Internet of things (IoT) data collection is an efficient and promising approach. The optimization of resource allocation in path planning is addressed in this paper by refining the energy consumption model and considering three metrics: the amount of collected data, time efficiency, and energy efficiency. The problem is formulated as a distributed partially observable Markov decision process (POMDP) and a novel deep reinforcement learning algorithm called RISE (Rényi state entropy)-D3QN (dueling double deep Q network) is proposed. It combines intrinsic rewards, prioritized experience replay, and soft-max exploration strategy, enabling path planning for UAV swarms while adapting to changes in UAV battery capacity, IoT device locations, data volume, and quantity. Simulation results demonstrate that compared to traditional D3QN and DQN algorithms, the proposed approach significantly increases. the amount of collected data from IoT devices while reducing UAV flight time and energy consumption, all while ensuring UAV safety during flight. [ABSTRACT FROM AUTHOR]

With the rapid development of unmanned aerial vehicles (UAVs) technologies, a substantial increase on the employ of UAV swarms in a wide range of civilian and military tasks has been witnessed. Advanced confrontation control approach can greatly improve UAVs’ capabilities and effectively free pilots from dangerous, boring, and burdensome confrontation missions. How to efficiently control UAV swarms in the air-to-air confrontation is still a hard problem. In this paper, considering the influence of the defending angle of UAV, we propose a general attacking cost function and an adaptive attacking strategy (AAS) to improve the capability of UAV swarm against another UAV swarm in an airborne battlefield. A multi-agent based UAV swarm air-to-air confrontation model is established, where red UAV swarm versus blue UAV swarm were simulated in a visual 3D and discrete-event environment. Extensive simulations are performed to verify the performance of AAS, the results show that AAS outperforms other traditional strategies by a large margin. In particular, UAV swarm adopting AAS can obtain a very high winning percentage even though the size of the swarm is only half of its opposing swarm that uses random or low velocity strategy. Meanwhile, AAS is quite robust to cope with different UAV swarm sizes. To improve the usability and practicability of AAS, we also propose a lightweight strategy called empirical adaptive attacking strategy (EAAS). The simulation results indicate that EAAS is easy to use and can retain the similar effects to AAS especially for large scale UAV swarms. Our work will illuminate new insights into the area of UAV swarm versus UAV swarm. [ABSTRACT FROM AUTHOR]

The Internet of vehicles (IoV) generates a massive amount of network connections and diversified data. To address the challenge that a single agent struggles to collect channel state information and perform service-differentiated resource allocation and link scheduling in dynamic scenarios, a multiagent deep reinforcement learning-based service-differentiated resource allocation method for IoV is proposed. This method aims to maximize the successful delivery rate of V2V link data packets and the total capacity of V2I links, under the constraint of minimizing interference to emergency service links. It employs deep reinforcement learning algorithms to optimize spectrum allocation and power selection strategies in a single-antenna vehicle-mounted network where multiple cellular users and device-to-device users coexist. Each agent is trained using deep Q-network(DQN), and they interact with the communication environment collectively, achieving coordination through a global reward function. Simulation results show that, in high-load scenarios, compared to traditional random allocation schemes, this scheme increases the total throughput of V2I links by 3.76 Mbps, improves the packet delivery rate of V2V links by 17.1%, and reduces the interference to emergency service links by 1.42 dB compared to ordinary links. This achieves priority guarantee for emergency service links and effectively enhances the overall transmission capacity of V2I and V2V links. [ABSTRACT FROM AUTHOR]

To further enhance the intelligent construction of coal mines and improve the control accuracy of hydraulic support displacement straightness, a digital twin control method for hydraulic support displacement has been proposed. First, a dataset related to hydraulic support is established, and a ridge regression prediction model is developed to achieve digital twin-based displacement decision analysis. Next, by analyzing the mechanical structure and displacement principles of the hydraulic supports, a hydraulic cylinder mathematics model is established, leading to the state-space representation of the controlled object. This study focuses on error control during the multi-agent operation of the hydraulic supports, designing a corresponding controller and using discretization methods to verify the consistency of output displacement between followers and leaders. Finally, simulation experiments based on the digital twin model of hydraulic supports are conducted, validating that the hydraulic supports can be controlled in formation according to actual production requirements. The digital twin control method enables the precise adaptive displacement control of hydraulic supports and provides valuable insights for the intelligent construction of mining faces. [ABSTRACT FROM AUTHOR]

A hierarchical consensus control algorithm based on value function decomposition is proposed for hierarchical multi-agent systems. To implement the consensus control algorithm, the reward function of the multi-agent systems can be decomposed, and two value functions can be obtained by analyzing the communication content and the corresponding control objective of each layer in the hierarchical multi-agent systems. Therefore, for each agent in the systems, a dual-critic network and a single-actor network structure are applied to realize the objective of each layer. In addition, the target network is introduced to prevent overfitting in the critic network and improve the stability of the online learning process. During the updating of network parameters, a soft updating mechanism and experience replay buffer are introduced to slow down the update rate of the network and improve the utilization rate of training data. The convergence and stability of the consensus control algorithm with the soft updating mechanism are analyzed theoretically. Finally, the correctness of the theoretical analysis and the effectiveness of the algorithm were verified by two experiments. [ABSTRACT FROM AUTHOR]

Current climate forecasting has advanced to the stage of investigating mesoscale air-sea interactions. Recent studies have identified significant structural differences between the cores and edges of mesoscale eddies; however, the effects of these structural variations on air-sea fluxes and the Marine Atmospheric Boundary Layer (MABL) remain underexplored. Traditional observations often fail to capture the detailed structures of eddies, necessitating enhanced observations at high spatiotemporal resolution for mesoscale eddies. To address this, efforts have been made to develop multi-agent platforms and expendable air-sea interface observation technologies. A task-oriented observation scheme was developed to monitor the spatial characteristics of mesoscale eddies. The South China Sea (SCS) is rich in mesoscale eddies with rapid motion changes, requiring enhanced observations of the air-sea interface using multi-agent mobile networking. An anticyclonic eddy was observed in the eastern region of the Xisha Islands in the SCS, and we examined variations in air-sea fluxes across different regions within the eddy. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence and unmanned technology, unmanned vehicles have been utilized in a variety of situations which may be hazardous to human beings, even in real battle fields. An intelligent unmanned vehicle can be aware of surrounding situations and make appropriate responding decisions. For this purpose, this paper applies Multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm for vehicle's of situation awareness and decision making, inside which a Fast Particle Swarm Optimization (FPSO) algorithm is proposed to calculate the optimal vehicle attitude and position; therefore, an improved deep reinforcement learning algorithm FPSO-MADPPG is formed. A specific advantage function is designed for the FPSO portion, which considers angle, distance, outflanking encirclement. A dedicated reward is designed for the MADPPG portion, which considers key factors like angle, distance, and damage. Finally, FPSO-MADPPG is then used in a combat game to operate unmanned tanks. Simulation results show that our method not only can obtain higher winning rate, but also higher reward and faster convergence than DDPG and MADPPG algorithms. [ABSTRACT FROM AUTHOR]

Multi-agent reinforcement learning is a varied and highly active field of research. The idea of parameter sharing or experience sharing has recently been introduced into multi-agent reinforcement learning to accelerate the training of multiple neural networks and enhance the final returns. However, implementing the parameter or experience sharing methods in multi-agent environments could introduce additional constraint or computation cost. This work presents a preference-based experience sharing scheme, which allows for different policies in environments with weakly homogeneous agents and requires barely any additional computational power. In this scheme, the experience replay buffer is augmented by adding a choice vector which indicates the preferred target of the agent, and each agent can learn various policies from the experience data collected by other agents who choose the same target. PSE-MADDPG, an off-policy algorithm with the preference-based experience sharing scheme, is proposed and benchmarked on a multi-target assignment and cooperative navigation mission. Experimental results show that PSE-MADDPG can successfully solve the problem of multiple targets assignment and outperform two classical deep reinforcement learning algorithms by learning in fewer steps and converging to higher episode rewards. Meanwhile, PSE-MADDPG relaxes the constraint of the strongly homogeneous agents assumption and requires little additional computation cost. [ABSTRACT FROM AUTHOR]

The strategy evolution process of game players is highly uncertain due to random emergent situations and other external disturbances. This paper investigates the issue of strategy interaction and behavioral decision-making among game players in simulated confrontation scenarios within a random interference environment. It considers the possible risks that random disturbances may pose to the autonomous decision-making of game players, as well as the impact of participants’ manipulative behaviors on the state changes of the players. A nonlinear mathematical model is established to describe the strategy decision-making process of the participants in this scenario. Subsequently, the strategy selection interaction relationship, strategy evolution stability, and dynamic decision-making process of the game players are investigated and verified by simulation experiments. The results show that maneuver-related parameters and random environmental interference factors have different effects on the selection and evolutionary speed of the agent’s strategies. Especially in a highly uncertain environment, even small information asymmetry or miscalculation may have a significant impact on decision-making. This also confirms the feasibility and effectiveness of the method proposed in the paper, which can better explain the behavioral decision-making process of the agent in the interaction process. This study provides feasibility analysis ideas and theoretical references for improving multi-agent interactive decision-making and the interpretability of the game system model. [ABSTRACT FROM AUTHOR]

Copyright of Integrated Intelligent Energy is the property of Editorial Department of Integrated Intelligent Energy and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In recent years, the study of enterprise survival development and cooperation in the whole economic market has been rapidly developed. However, in most literature studies, the traditional enterprise multi-agent cannot effectively simulate the process of enterprise survival and development since the fundamental characteristics used to describe enterprises in social networks, such as the values of enterprise multi-agent attributes, cannot be changed in process of the simulation. To address this problem, an enterprise multi-agent model based on game Q- learning to simulate enterprise decision making which aims to maximize the benefits of enterprises and optimize the effect of inter-firm cooperation is proposed in this article. The Firm Q Learning algorithm is used to dynamically change the attribute values of the enterprise multi-agent to optimize the game results in the evolutionary game model and thus effectively simulate the dynamic cooperation among the enterprise agents. The simulation result shows that the evolution of the enterprise multi-agent model based on game Q-learning can more realistically reflect the process of real enterprise survival and development than the multi-agent simulation with fixed attribute values. [ABSTRACT FROM AUTHOR]

In this paper, we propose the problem of connectivity maintenance for a group of robots that are connected with wireless communication. The communication between the agents is modeled by a fading channel, and the exchange of information is possible at permissible distances. The distributed controller is designed to maintain the global connectivity of the network communication graph. On the other hand, obtaining the best communication quality between agents is the definition of good network connectivity. Therefore, the Laplacian matrix is defined as a weighted graph according to the communication parameters. Initially, the controller uses the supergradation algorithm to remake the network Laplacian matrix for having a bigger second eigenvalue. While each agent in the network only has access to the neighbor's information. The control algorithm uses a multi-agent estimator to find the eigenvector corresponding to the second small eigenvalue. The formation problem has been considered in the second step after the reference topology has been obtained. Because of the nonlinear dynamics of the agents, the sliding mode controller has been used for this purpose. This robust controller could be a suitable choice due to the modeling uncertainty and sensor measurement uncertainty. Finally, an example of multi-agent robots is provided to evaluate the algorithm.

In today's rapidly evolving manufacturing landscape, the efficient allocation of human resources is crucial for optimizing production systems. This paper addresses the dual resource-constrained flexible job shop scheduling problem (DRCFJSP) by implementing a Multi-Start Tabu Search-based Multi-Agent Model (MuSTAM). MuSTAM considers a set of initial solutions that are run in parallel using an intensification technique. The main objective of MuSTAM is to reduce the completion time (makespan), which is an important metric in workshops. The proposed model is composed of two classes of agents: a supervisor agent named MainAgent and a set of tabu search agents called TabuAgents. The MainAgent launches the system, collects inputs, generates the initial population, creates TabuAgents based on the number of solutions in the initial population (PopSize), and displays the best solution. Each TabuAgent receives a solution from the initial population and then conducts a concentrated search around its neighbourhood space by applying the intensification technique of tabu search. TabuAgents cooperate and communicate with each other to enhance search quality. Numerical tests are performed during the experimental phase to compare MuSTAM with ITS based on well-known benchmark instances of the DRCFJSP. The results demonstrate that MuSTAM is effective in terms of makespan and CPU time.

The paper presents a simulation investigation into the individual time of parcel transportation in a multi-agent robot operated warehouse for two different artificial intelligence planning strategies (independent and cooperative), two different warehouse setups and different number of robots operating at the same time. The robots are assumed to operate in a non-communicating manner, and their only physical interactions are the collision detection and avoidance maneuvers. It is additionally assumed, that every robot is aware of its location, is able to compute the shortest path to the required destination and has access to a global orders list from which orders can be reserved and removed. The warehouse setup and robots movement is simulated using a cellular automata paradigm, and the simulation is implemented in Python. It has been shown that the cooperation of robots passing packages to each other significantly improves the overall work time needed to complete the task and reduces the number of empty runs compared to an individual approach. The presented simulation results suggest it is possible to design a multi-robot delivery system in such a way, that robots do not communicate with each other and that no central planning or global optimization center is needed. The novel cooperative planning algorithm brings a contribution in engineering solutions, mainly for robot development in automated warehouses. [ABSTRACT FROM AUTHOR]

For the timeliness of millimeter-wave secure beamforming in mobile scenarios and the robustness of secure connections in millimeter-wave dynamic eavesdropping scenarios, a multi-agent secure cooperative communication scheme based on a dueling double deep Q network (D3QN)-deep deterministic policy gradient (DDPG) algorithm was proposed to address communication security issues. The scheme utilized road side unit (RSU)-assisted cooperative jamming technology to reduce the eavesdropper’s reception quality of confidential signals. The optimization problem was formulated to maximize the total secrecy rate of all legitimate vehicles by optimizing the joint base station and beam connection control of the VUs, the selection of the jamming RSUs, and the cooperative jamming direction and transmit power of RSUs. Furthermore, for the challenges posed by the high dynamics of vehicular networks, the scheme achieved a fusion of real-time discrete and continuous decision-making by creating a VU agent grounded in D3QN and an RSU agent harnessing D3QN-DDPG capabilities. Finally, through multi-dimensional performance analysis and scheme comparison experiments, simulation results demonstrate the effectiveness of the proposed scheme.

Most optimisation research focuses on relatively simple cases: one decision maker, one objective, and possibly a set of constraints. However, real-world optimisation problems often come with complications: they might be multi-objective, multi-agent, multi-stage or multi-level, and they might have uncertainty, partial knowledge or nonlinear objectives. Each has led to research areas with dedicated solution methods. However, when new hybrid problems are encountered, there is typically no solver available. We define a broad class of discrete optimisation problem called an influence program, and describe a lightweight algorithm based on multi-agent multi-objective reinforcement learning with sampling. We show that it can be used to solve problems from a wide range of literatures: constraint programming, Bayesian networks, stochastic programming, influence diagrams (standard, limited memory and multi-objective), and game theory (multi-level programming, Bayesian games and level-k reasoning). We expect it to be useful for the rapid prototyping of solution methods for new hybrid problems.

In this paper, we propose a battery management algorithm to maximize the lifetime of a parallel-series connected battery pack with heterogeneous states of health in a battery energy storage system. The growth of retired lithium-ion batteries from electric vehicles increases the applications for battery energy storage systems, which typically group multiple individual batteries with heterogeneous states of health in parallel and series to achieve the required voltage and capacity. However, previous work has primarily focused on either parallel or series connections of batteries due to the complexity of managing diverse battery states, such as state of charge and state of health. To address the scheduling in parallel-series connections, we propose a cooperative multi-agent deep Q network framework that leverages multi-agent deep reinforcement learning to observe multiple states within the battery energy storage system and optimize the scheduling of cells and modules in a parallel-series connected battery pack. Our approach not only balances the states of health across the cells and modules but also enhances the overall lifetime of the battery pack. Through simulation, we demonstrate that our algorithm extends the battery pack's lifetime by up to 16.27% compared to previous work and exhibits robustness in adapting to various power demand conditions. [ABSTRACT FROM AUTHOR]

Due to the advantages of a centralized critic to estimate the Q-function value and decentralized actors to optimize the agents' policies, counterfactual multi-agent (COMA) stands out in most multi-agent reinforcement learning (MARL) algorithms. The sharing of policy parameters can improve sampling efficiency and learning effectiveness, but it may lead to a lack of policy diversity. Hence, to balance parameter sharing and diversity among agents in COMA has been a persistent research topic. In this paper, an effective training method for a COMA policy network based on a differential evolution (DE) algorithm is proposed, named DE-COMA. DE-COMA introduces individuals in a population as computational units to construct the policy network with operations such as mutation, crossover, and selection. The average return of DE-COMA is set as the fitness function, and the best individual of policy network will be chosen for the next generation. By maintaining better parameter sharing to enhance parameter diversity, multi-agent strategies will become more exploratory. To validate the effectiveness of DE-COMA, experiments were conducted in the StarCraft II environment with 2s_vs_1sc, 2s3z, 3m, and 8m battle scenarios. Experimental results demonstrate that DE-COMA significantly outperforms the traditional COMA and most other multi-agent reinforcement learning algorithms in terms of win rate and convergence speed. [ABSTRACT FROM AUTHOR]

Mobile Edge Computing (MEC) is crucial for reducing latency by bringing computational resources closer to the network edge, thereby enhancing the quality of services (QoS). However, the broad deployment of cloudlets poses challenges in efficient network slicing, particularly when traffic distribution is uneven. Therefore, these challenges include managing diverse resource requirements across widely distributed cloudlets, minimizing resource conflicts and delays, and maintaining service quality amid fluctuating request rates. Addressing this requires intelligent strategies to predict request types (common or urgent), assess resource needs, and allocate resources efficiently. Emerging technologies like edge computing and 5G with network slicing can handle delay-sensitive IoT requests rapidly, but a robust mechanism for real-time resource and utility optimization remains necessary. To address these challenges, we designed an end-to-end network slicing approach that predicts common and urgent user requests through T distribution. We formulated our problem as a multi-agent Markov decision process (MDP) and introduced a multi-agent soft actor–critic (MAgSAC) algorithm. This algorithm prevents the wastage of scarce resources by intelligently activating and deactivating virtual network function (VNF) instances, thereby balancing the allocation process. Our approach aims to optimize overall utility, balancing trade-offs between revenue, energy consumption costs, and latency. We evaluated our method, MAgSAC, through simulations, comparing it with the following six benchmark schemes: MAA3C, SACT, DDPG, S2Vec, Random, and Greedy. The results demonstrate that our approach, MAgSAC, optimizes utility by 30%, minimizes energy consumption costs by 12.4%, and reduces execution time by 21.7% compared to the closest related multi-agent approach named MAA3C. [ABSTRACT FROM AUTHOR]

In the context of China's swift urbanization and heightened sustainability challenges, SMCEs in the construction industry play a critical role in driving both economic growth and sustainable development. This pioneering study introduces an integrative approach blending sustainability-oriented strategies (SSs) with financing guarantees, addressing key obstacles faced by these firms. Leveraging a multi-agent computational model, we dissect the dynamic interplay among construction SMCEs, governmental bodies, financial institutions, and guarantors, underscoring the transformative impact of financing guarantees on sustainable advancement. Our investigation reveals that optimally structured financing guarantees not only resolve funding challenges but also actively foster adherence to sustainability practices. This dual functionality positions such financial instruments as key enablers of SMCEs' quest for sustainability, aligning economic objectives with ecological imperatives. Furthermore, we spotlight the transformative effect of strategic management on elevating SMCEs' commitment to sustainable operations. Collectively, these insights offer actionable pathways for SMCEs embarking on sustainability endeavors, showcasing innovative financial solutions that do not detract from their economic prosperity. [ABSTRACT FROM AUTHOR]

Task offloading and resource allocation is a research hotspot in cloud-edge collaborative computing. Many existing pieces of research adopted single-agent reinforcement learning to solve this problem, which has some defects such as low robustness, large decision space, and ignoring delayed rewards. In view of the above deficiencies, this paper constructs a cloud-edge collaborative computing model, and related task queue, delay, and energy consumption model, and gives joint optimization problem modeling for task offloading and resource allocation with multiple constraints. Then, in order to solve the joint optimization problem, this paper designs a decentralized offloading and scheduling scheme based on "task-oriented" multi-agent reinforcement learning. In this scheme, we present information synchronization protocols and offloading scheduling rules and use edge servers as agents to construct a multi-agent system based on the Actor–Critic framework. In order to solve delayed rewards, this paper models the offloading and scheduling problem as a "task-oriented" Markov decision process. This process abandons the commonly used equidistant time slot model but uses dynamic and parallel slots in the step of task processing time. Finally, an offloading decision algorithm TOMAC-PPO is proposed. The algorithm applies the proximal policy optimization to the multi-agent system and combines the Transformer neural network model to realize the memory and prediction of network state information. Experimental results show that this algorithm has better convergence speed and can effectively reduce the service cost, energy consumption, and task drop rate under high load and high failure rates. For example, the proposed TOMAC-PPO can reduce the average cost by from 19.4% to 66.6% compared to other offloading schemes under the same network load. In addition, the drop rate of some baseline algorithms with 50 users can achieve 62.5% for critical tasks, while the proposed TOMAC-PPO only has 5.5%. [ABSTRACT FROM AUTHOR]

Damming can promote flood control, water supply, power generation, and shipping but often changes the downstream hydrological regimes, producing adverse externality effects. Maximizing downstream social and ecological benefits will inevitably reduce upstream power generation. This study presents two novel bi-level hybrid game models, called the non-cooperative hybrid game model (NCHG) and the cooperative hybrid game model (CHG), to facilitate integrated water resource management in reservoir systems. The performance index of propensity to disrupt is applied to evaluate the stability of CHG, and an improved reliability index and Gini coefficient are adopted to evaluate the reliability and equity of both two models. The Three Gorges Reservoir and its adjacent cities were chosen as a case to inspect the two models' performance. A range of scheduling schemes was derived by proposed bi-level hybrid game models in wet, normal, and dry years. Results reveal that (i) the RI values of the watershed system obtained by the CHG are less than those in NCHG in three typical years (for example, 0.1201 VS 0.1930 in the wet year), showing higher systemic reliability. The Gini coefficients of the watershed system obtained by the CHG are all less than those obtained by the NCHG in all typical years (for example, 0.1016 VS 0.1020 in the wet year), which shows better performance of CHG on fairness for the allocation results; (ii) in the case of a multi-function reservoir system, the CHG generates favorable allocation schemes with higher systemic characteristic values by 32.43, 34.39, and 33.54 in wet, normal, and dry years, respectively, than those in NCHG (32.03, 33.16, and 31.42 in wet, normal, and dry years, respectively); (iii) compared with NCHG, the economic benefits obtained by CHG decreased by 0.98%, 1.04%, and 5.42% in wet, normal, and dry years, respectively; the social negative benefits decreased by 3.49%, 9.84%, and 28.69%; and the ecological negative benefits decreased by 1.77%, 5.65%, and 5.59%, respectively. It indicates that a minor sacrifice of the reservoir benefit could significantly improve the welfare at the system level by the CHG. The developed CHG can provide optimal water scheduling schemes in balancing inter-regional water conflicts and can be widely used to produce an equilibrium management strategy for a multi-function reservoir system. [ABSTRACT FROM AUTHOR]

The emergence of beyond 5G networks has the potential for seamless and intelligent connectivity on a global scale. Network slicing is crucial in delivering services for different, demanding vertical applications in this context. Next-generation applications have time-sensitive requirements and depend on the most efficient routing path to ensure packets reach their intended destinations. However, the existing IP (Internet Protocol) over a multi-domain network faces challenges in enforcing network slicing due to minimal collaboration and information sharing among network operators. Conventional inter-domain routing methods, like Border Gateway Protocol (BGP), cannot make routing decisions based on performance, which frequently results in traffic flowing across congested paths that are never optimal. To address these issues, we propose CoopAI-Route, a multi-agent cooperative deep reinforcement learning (DRL) system utilizing hierarchical software-defined networks (SDN). This framework enforces network slicing in multi-domain networks and cooperative communication with various administrators to find performance-based routes in intra- and inter-domain. CoopAI-Route employs the Distributed Global Topology (DGT) algorithm to define inter-domain Quality of Service (QoS) paths. CoopAI-Route uses a DRL agent with a message-passing multi-agent Twin-Delayed Deep Deterministic Policy Gradient method to ensure optimal end-to-end routes adapted to the specific requirements of network slicing applications. Our evaluation demonstrates CoopAI-Route's commendable performance in scalability, link failure handling, and adaptability to evolving topologies compared to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Guangdong University of Technology is the property of Journal of Guangdong University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Aiming at the salvo-attack problem of multiple missiles, a distributed cooperative guidance law based on the event-triggered mechanism is proposed, which enables missiles with large differences in spatial location and velocity to achieve simultaneous attacks with only a few dozen information exchanges. It effectively reduces the generation of control commands and communication frequency, thereby reducing channel load and improving communication efficiency and reliability. Compared to traditional periodic sampling communication, the number of communications has been reduced by over 90 % . The guidance process is divided into two stages. The first stage is the cooperative guidance stage, where missiles achieve consensus of the time-to-go estimates through information exchange. In this stage, each missile is designed with an event-triggered function based on its own state error, and the missile only updates and transmits its information in the communication network when the error meets the set threshold, effectively reducing the occupancy rate of missile-borne resources during the cooperation process. The second stage is the independent guidance stage, where missiles can hit the target simultaneously while keeping the communication network silent. This is achieved by ensuring that the time-to-go estimates of missiles can represent the real time-to-go after achieving consensus. By the design of the two-stage guidance law and the replacement of the event-triggered function, the cooperative guidance system can be ensured to remain stable in scenarios where the leader missile is present and destroyed, and this excludes Zeno behavior. The stability of the cooperative guidance law is rigorously proved by algebraic graph theory, matrix theory, and the Lyapunov method. Finally, the numerical simulation results demonstrate the validity of the algorithm and the correctness of the stability analysis. [ABSTRACT FROM AUTHOR]

Most optimisation research focuses on relatively simple cases: one decision maker, one objective, and possibly a set of constraints. However, real-world optimisation problems often come with complications: they might be multi-objective, multi-agent, multi-stage or multi-level, and they might have uncertainty, partial knowledge or nonlinear objectives. Each has led to research areas with dedicated solution methods. However, when new hybrid problems are encountered, there is typically no solver available. We define a broad class of discrete optimisation problem called an influence program, and describe a lightweight algorithm based on multi-agent multi-objective reinforcement learning with sampling. We show that it can be used to solve problems from a wide range of literatures: constraint programming, Bayesian networks, stochastic programming, influence diagrams (standard, limited memory and multi-objective), and game theory (multi-level programming, Bayesian games and level-k reasoning). We expect it to be useful for the rapid prototyping of solution methods for new hybrid problems. [ABSTRACT FROM AUTHOR]

Cooperative multi-agent systems make it possible to employ miniature robots in order to perform different experiments for data collection in wide open areas to physical interactions with test subjects in confined environments such as a hive. This paper proposes a new multi-agent path-planning approach to determine a set of trajectories where the agents do not collide with each other or any obstacle. The proposed algorithm leverages a risk-aware probabilistic roadmap algorithm to generate a map, employs node classification to delineate exploration regions, and incorporates a customized genetic framework to address the combinatorial optimization, with the ultimate goal of computing safe trajectories for the team. Furthermore, the proposed planning algorithm makes the agents explore all subdomains in the workspace together as a formation to allow the team to perform different tasks or collect multiple datasets for reliable localization or hazard detection. The objective function for minimization includes two major parts, the traveling distance of all the agents in the entire mission and the probability of collisions between the agents or agents with obstacles. A sampling method is used to determine the objective function considering the agents' dynamic behavior influenced by environmental disturbances and uncertainties. The algorithm's performance is evaluated for different group sizes by using a simulation environment, and two different benchmark scenarios are introduced to compare the exploration behavior. The proposed optimization method establishes stable and convergent properties regardless of the group size. [ABSTRACT FROM AUTHOR]

Cross-domain cooperative task allocation is a complex and challenging issue in the field of multi-agent task allocation that requires urgent attention. This paper proposes a task allocation method based on the multi-strategy improved dung beetle optimization (MSIDBO) algorithm, aiming to solve the problem of fully distributed multi-agent cross-domain cooperative task allocation. This method integrates two key objective functions: target allocation and control allocation. We propose a target allocation model based on the optimal comprehensive efficiency, cluster load balancing, and economic benefit maximization, and a control allocation model leveraging the radar detection ability and control data link connectivity. To address the limitations of the original dung beetle optimization algorithm in solving such problems, four revolutionary strategies are introduced to improve its performance. The simulation results demonstrate that our proposed task allocation algorithm significantly improves the cross-domain collaboration efficiency and meets the real-time requirements for multi-agent task allocation on various scales. Specifically, our optimization performance was, on average, 32.5% higher compared to classical algorithms like the particle swarm optimization algorithm and the dung beetle optimization algorithm and its improved forms. Overall, our proposed scheme enhances system effectiveness and robustness while providing an innovative and practical solution for complex task allocation problems. [ABSTRACT FROM AUTHOR]

In target-oriented multi-agent tasks, agents collaboratively achieve goals defined by specific objects, or targets, in their environment. The key to success is the effective coordination between agents and these targets, especially in dynamic environments where targets may shift. Agents must adeptly adjust to these changes and re-evaluate their target interactions. Inefficient coordination can lead to resource waste, extended task times, and lower overall performance. Addressing this challenge, we introduce the regulatory hierarchical multi-agent coordination (RHMC), a hierarchical reinforcement learning approach. RHMC divides the coordination task into two levels: a high-level policy, assigning targets based on environmental state, and a low-level policy, executing basic actions guided by individual target assignments and observations. Stabilizing RHMC's high-level policy is crucial for effective learning. This stability is achieved by reward regularization, reducing reliance on the dynamic low-level policy. Such regularization ensures the high-level policy remains focused on broad coordination, not overly dependent on specific agent actions. By minimizing low-level policy dependence, RHMC adapts more seamlessly to environmental changes, boosting learning efficiency. Testing demonstrates RHMC's superiority over existing methods in global reward and learning efficiency, highlighting its effectiveness in multi-agent coordination. [ABSTRACT FROM AUTHOR]

Construction engineering and management (CEM) involves multiple stakeholders, complex interest relationships, and conflicts. All stakeholders must comprehensively consider the interests relating to a project to make decisions. Appropriate multi-agent decision-making can effectively coordinate and integrate the needs or expectations of all stakeholders, which can reduce conflicts, improve the success probability of the project, maximize the overall returns on interest, and contribute to the project's sustainability. Existing studies have begun to explore the associated theoretical framework and practical methods for multi-agent decision-making. However, early studies mainly focused on the conceptual theories of decision-making models and processes, such as stakeholder analysis, decision evaluation, and risk management. In recent years, increasing research has concerned the application of multi-agent decision-making in CEM. This paper is the first to use a comprehensive review approach to analyze multi-agent decision-making in CEM, providing an overall perspective. In this paper, 105 journal papers are identified and classified into four categories: (1) main concerns regarding multi-agent decision-making in CEM; (2) tools and methods of multi-agent decision-making in practice; (3) research tools and methods of multi-agent decision-making in CEM; (4) critical points on how to solve multi-agent decision-making problems. The findings of this study offer references for future trends in four regards, namely (1) knowledge management, (2) decision resilience, (3) the fusion of many techniques, and (4) technologies for future research. [ABSTRACT FROM AUTHOR]

Traditional multi-agent path planning algorithms often lead to path overlap and excessive energy consumption when dealing with cooperative tasks due to the single-agent-single-task configuration. For this reason, the "many-to-one" cooperative planning method has been proposed, which, although improved, still faces challenges in the vast search space for meeting points and unreasonable task handover locations. This paper proposes the Cooperative Dynamic Priority Safe Interval Path Planning with a multi-meeting-point and single-meeting-point solving mode switching (Co- D P S I P P m s ) algorithm to achieve multi-agent path planning with task handovers at multiple or single meeting points. First, the initial priority is set based on the positional relationships among agents within the cooperative group, and the improved Fermat point method is used to locate multiple meeting points quickly. Second, considering that agents must pick up sub-tasks or conduct task handovers midway, a segmented path planning strategy is proposed to ensure that cooperative agents can efficiently and accurately complete task handovers. Finally, an automatic switching strategy between multi-meeting-point and single-meeting-point solving modes is designed to ensure the algorithm's success rate. Tests show that Co- D P S I P P m s outperforms existing algorithms in 1-to-1 and m-to-1 cooperative tasks, demonstrating its efficiency and practicality. [ABSTRACT FROM AUTHOR]

Most of the existing MAPF methods based on communication learning have poor scalability or aggregate too much redundant information, resulting in inefficient communication. To solve these problems, this paper proposed disruptor identifiable communication (DIC), which learned concise communication excluding non-disruptors by judging whether the agent in the center of the field of view would change its decision-making due to the presence of neighbors, and successfully filtered out redundant information. At the same time, this paper further instantiated DIC and developed a new highly scalable distributed MAPF solver: disruptor identifiable communication based on reinforcement and imitation learning algorithm (DICRIA). Firstly, the disruptor discriminator and the policy output layer of DICRIA identified the disruptor. Secondly, the algorithm updated the information of the disruptor and the communication wish sender in two rounds of communication respectively. Finally, DICRIA output the final policy according to the coding results of each module. Experimental results show that DICRIA S performance is better than other similar solvers in almost all environment settings, and the algorithm increases the success rate by 5.2% on average compared to the baseline solver. Especially in dense problem instances with large-size maps, the algorithm even increases the success rate of DICRIA by 44.5% compared to the baseline solver. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Engineering Management / Gongcheng Guanli Xuebao is the property of Journal of Engineering Management Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Shanghai Jiaotong University (Science) is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Chongqing University of Technology (Natural Science) is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In this study, dynamic formation tracking and interception are performed by controlling multi-agent using a Euler-like Lagrangian model. The purpose is to use the distance-based rigid graph method to control multi-agent, and ultimately achieve dynamic formation tracking and target interception of multi-agent. Initially, distance-based graph stiffness and back-stepping techniques were considered to address the formation control challenge. This method helps achieve the initial expected formation and effectively complete the formation mission. Leaders continue to chase the moving target, while followers stick to the expected arrangement. The leader then precisely tracks the moving target and surrounds it within the formation. By using Lyapunov stability theory with adaptive control, it is ensured that the total distance meets finite and consistent error limits. Finally, the numerical simulation of the interception plan was carried out by 6 multi-agents and 1 target at different times to verify the effectiveness of the control method. [ABSTRACT FROM AUTHOR]