Search

Global Nash equilibrium is an optimal solution for each player in a graphical game. This paper proposes an iterative adaptive dynamic programming-based algorithm to solve the global Nash equilibrium solution for optimal containment control problem with robustness analysis to the iterative error. The containment control problem is transferred into the graphical game formulation. Sufficient conditions are given to decouple the Hamilton–Jacobi equations, which guarantee the solvability of the global Nash equilibrium solution. The iterative algorithm is designed to obtain the solution without any knowledge of system dynamics. Conditions of iterative error for global stability are given with rigorous proof. Compared with existing works, the design procedures of control gain and coupling strength are separated, which avoids trivial cases in the design procedure. The robustness analysis exactly quantifies the effect of the iterative error caused by various sources in engineering practice. The theoretical results are validated by two numerical examples with marginally stable and unstable dynamics of the leader. [ABSTRACT FROM AUTHOR]

• Developed a PPRL method for satellite formation control under multi-constraint. • Modeled dynamics, state, performance, and interference constraints within RL framework. • Implemented a two-step training strategy for dual-satellite coordination control. • Enhanced the adaptability and efficiency of satellite formation control. As the complexity of space missions increases, the constraints on satellite attitude control become more stringent, particularly for satellites working in orbit formation. This paper introduces a novel method, based on the categorization and modeling of different constraints, for attitude control of satellite formations under multiple constraints. The method employs the Phased Priority Reinforcement Learning (PPRL) approach, which utilizes Deep Deterministic Policy Gradient (DDPG) technology. Considering the complexity of constraints and the challenge posed by the high control dimensionality due to multi-satellite coordination, the method addresses these challenges through a two-step training strategy. The first step addresses the multi-constraint issue for individual satellites and increases the priority of single-satellite training experience data in the experience replay buffer of the second step to enhance data utilization efficiency. To address the issue of reward sparsity in complex high-dimensional constraint models, a detailed reward mechanism is proposed, incorporating both local and global constraints into the reward function, thereby achieving both efficient and effective attitude control. This approach not only meets dynamic, state, and performance constraints but also demonstrates adaptability and robustness through numerical simulations. Compared to traditional methods, this approach achieves significant improvements in control performance and constraint satisfaction, offering a novel solution pathway for high-dimensional control problems in multi-constraint satellite formations. [ABSTRACT FROM AUTHOR]

This paper is concerned with the secure output consensus problem for the heterogeneous multi-agent systems under the event-triggered scheme in the presence of the denial-of-service attack. Without detecting the attack, the hold-input controller update strategy is adopted when some transmission data may be lost due to the effect of the attack. Based on the tolerable duration of the attack, a novel edge-based event-triggered scheme is developed. The scheme can avoid continuous communication and exclude Zeno behavior. With the aid of the switched system theory, output consensus is preserved. An example shows the effectiveness. • Secure problem for the heterogeneous multi-agent systems is solved in the presence of the denial-of-service attack. • A novel edge-based event-triggered scheme is developed based on the tolerable duration of the attack. • Without detecting the attack, the hold-input controller update strategy is adopted. • Output consensus is preserved with the aid of the switched system theory. [ABSTRACT FROM AUTHOR]

Research efforts focused on Smart Building (SB) development have concentrated on the automation of resources within intelligent environments towards an enhanced experience for occupants. The process converts everyday manual activities into automatic actions, such as turning on lights upon entering a room, activating air conditioning on hot days, and switching off a television when there are no viewers. This article addresses a case study on context-aware monitoring conducted at the Laboratory of Distributed Systems and Concurrent Programming (LaSDPC) of the University of São Paulo. The focus is on a Fog layer of the IoT operating closer to sensors and reducing communication delays. Concepts of context-aware systems, multi-agent systems, ontology, and MQTT protocol were considered for the implementation of the intelligent system on-site. The programming languages used were Java and Python, leveraging libraries and frameworks dedicated to such technologies. A scalable system that does not compromise computational resource use and maintains responsiveness with low data exchange latency is proposed and tests checked the intelligent behavior of the laboratory under specific conditions using temperature, luminosity, and presence of a person as parameters. [ABSTRACT FROM AUTHOR]

• Shared camera measurements in a dual-agent system for pose estimation using a MEKF. • Stereo-vision measurements remove range ambiguity for relative pose estimation. • Relative navigation estimation accuracy improves for proximity operations using stereo-vision measurements. This paper presents a novel autonomous relative navigation architecture for inspector spacecraft in proximity operations to a known structure in a lunar orbit. Autonomous on-orbit servicing and assembly of large space structures are at the forefront of research for advancing space exploration beyond low-earth orbit. Communication delays with ground stations and limited computational power located on the inspector spacecraft make current relative navigation methods unfeasible and in turn, make it difficult to maintain an accurate estimate of the relative position and orientation of the inspector spacecraft with respect to the structure. The proposed method utilizes the camera measurements from two identical inspector spacecraft, each equipped with a single camera, to estimate the relative position and orientation of the structure. This shared measurement strategy overcomes the deficiencies in monocular vision-based relative navigation techniques, such as the range ambiguities, as is proven in the simulated results of this paper. The proposed stereo-vision system in this work allows for a deterministic solution of the relative position and orientation from the measurements of each inspector spacecraft to the structure. The stereo-vision system was tested with a Multiplicative Extended Kalman Filter (MEKF) for each spacecraft to estimate the relative states of the structure while maintaining a fixed relative position to the structure. The performance of the proposed stereo-vision system was evaluated in terms of accuracy and reliability of the estimated relative position and orientation of two inspector spacecraft to the structure over 100 Monte Carlo simulations for three case studies. The results showed that the stereo-vision system significantly outperforms the monocular vision system even when fewer points on the structure are visible, as is the case for proximity operations. Additionally, the stereo-vision system is consistently more accurate for the relative velocity, attitude, and angular velocity states regardless of how many points are visible. The proposed stereo-vision relative navigation system has the potential to be applied autonomously in proximity operations of spacecraft for various space missions, such as rendezvous and docking, inspection, and on-orbit servicing. [ABSTRACT FROM AUTHOR]

This paper considers a class of multi-agent distributed convex optimization with a common set of constraints and provides several continuous-time neurodynamic approaches. In problem transformation, l 1 and l 2 penalty methods are used respectively to cast the linear consensus constraint into the objective function, which avoids introducing auxiliary variables and only involves information exchange among primal variables in the process of solving the problem. For nonsmooth cost functions, two differential inclusions with projection operator are proposed. Without convexity of the differential inclusions, the asymptotic behavior and convergence properties are explored. For smooth cost functions, by harnessing the smoothness of l 2 penalty function, finite- and fixed-time convergent algorithms are provided via a specifically designed average consensus estimator. Finally, several numerical examples in the multi-agent simulation environment are conducted to illustrate the effectiveness of the proposed neurodynamic approaches. [ABSTRACT FROM AUTHOR]

In times of increasingly complex production processes and volatile customer demands, the production adaptability is crucial for a company's profitability and competitiveness. The ability to cope with rapidly changing customer requirements and unexpected internal and external events guarantees robust and efficient production processes, requiring a dedicated control concept at the shop floor level. Yet in today's practice, conventional control approaches remain in use, which may not keep up with the dynamic behaviour due to their scenario-specific and rigid properties. To address this challenge, deep learning methods were increasingly deployed due to their optimization and scalability properties. However, these approaches were often tested in specific operational applications and focused on technical performance indicators such as order tardiness or total throughput. In this paper, we propose a deep reinforcement learning based production control to optimize combined techno-financial performance measures. Based on pre-defined manufacturing modules that are supplied and operated by multiple agents, positive effects were observed in terms of increased revenue and reduced penalties due to lower throughput times and fewer delayed products. The combined modular and multi-staged approach as well as the distributed decision-making further leverage scalability and transferability to other scenarios. [ABSTRACT FROM AUTHOR]

This article discusses self-triggering algorithm using distributed model predictive control (DMPC) to achieve dynamic consensus in linear multi-agent systems (MASs). The iterative computations and communications required at each time step in traditional consensus algorithms cause escalation of the energy consumption and shorten the life span of the MAS. An attempt to solve this problem is made by proposing a sequential self-triggering consensus algorithm, where each agent computes its own triggering instants. A Laguerre based DMPC design is adopted that notably reduces the computational complexity of conventional DMPC. The proposed self-triggered DMPC algorithm optimizes the control input and triggering interval while guaranteeing the dynamic consensus of the agents. By virtue of the Laguerre function based control architecture, the additional computations owing to the self-triggered algorithm do not impose stress on the controller; yet reduce the load on communication resources. The equality constraint on the terminal state of the agents is utilized along with Lyapunov criteria to establish the closed loop stability of the MAS. The proposed scheme achieves a considerable drop in controller design computations as well as data transmissions among agents, and the same is established by comparing these traits of existing schemes while achieving comparable performance. The proposed algorithm is verified through simulation of platoon configuration of vehicles, each of which is modeled as a linear multi-input multi-output (MIMO) system. • A discrete Laguerre based self triggered control design is proposed for dynamic consensus of multi-agent system • The proposed design directly computes the optimal control input and triggering instants for each agent • The design achieves dynamic consensus with reduced data transmissions for multi-agents [ABSTRACT FROM AUTHOR]

Effective exploration is the key to achieving high returns for reinforcement learning. Agents must explore jointly in multi-agent systems to find the optimal joint policy. Due to the exploration problem and the shared reward, the policy-based multi-agent reinforcement learning algorithms face policy overfitting, which may lead to the joint policy falling into a local optimum. This paper introduces a novel general framework called Learning Joint-Action Intrinsic Reward (LJIR) for improving multi-agent reinforcement learners' joint exploration ability and performance. LJIR observes agents' state and joint actions to learn to construct an intrinsic reward online that can guide effective joint exploration. With the novel combination of Transformer and random network distillation, LJIR selects the novel states to give more intrinsic rewards, which help agents find the best joint actions. LJIR can dynamically adjust the weight of exploration and exploitation during training and keep the policy invariance finally. To ensure LJIR seamlessly adopts existing MARL algorithms, we also provide a flexible combination method for intrinsic and external rewards. Empirical results on the SMAC benchmark show that the proposed method achieves state-of-the-art performance in challenging tasks. [ABSTRACT FROM AUTHOR]

In a socialized multi-agent system, the preferences of individuals will be inevitably influenced by others. This paper introduces an extended empathy structure to characterize the coupling process of preferences under specific relations and make it cover scenarios including human society, human–machine system, and even abiotic engineering applications. In this model, empathy is abstracted as a stochastic experience process in the form of Markov chain, and the coupled empathy utility is defined as the expectation of obtaining preferences under the corresponding probability distribution. The self-other separation is the core concept with which our structure can exhibit social attributes, including attraction of implicit states, inhibition of excessive empathy, attention of empathetic targets, and anisotropy of the utility distribution. Compared with the previous empirical models, our model has a better performance on the data set and can provide a new perspective for designing and analyzing the cognitive layer of the human–machine network, as well as the information fusion and semi-supervised clustering methods in engineering. [ABSTRACT FROM AUTHOR]

To enable resilient manufacturing, production control systems must be able to adapt quickly and flexibly to unexpected changes. Motivated by Industrie 4.0, multi-agent systems offer a promising approach to meet these requirements. This paper presents a method for distributed production control that enables production entities to make optimal decisions regarding capacity utilization and the compliance of delivery dates in real-time despite disruptive events. Based on a multi-agent Markov Decision Process, production entities are modeled as autonomous software agents that can solve Flexible Job-Shop Problems through message passing. The performance of the proposed method is evaluated in a simulated production environment. [ABSTRACT FROM AUTHOR]

In this paper, the model-free formation control problem is addressed for multiple satellites, whose dynamics involves rotation-translation coupling and nonlinearities. A two-step adaptive control strategy is proposed based on the reinforcement learning theory to iteratively solve the optimal formation control problem without knowledge of each satellite dynamics. The resulted controller is composed of an attitude controller to achieve the desired attitude, and a relative position controller to construct the satellite formation. The proposed global closed-loop control system is proven to be asymptotically stable. Simulation results on rotation-translation coupled satellite system confirm the effectiveness of the proposed adaptive optimal formation control strategy. • The model-free formation control problem is addressed for multiple satellites. • The dynamics of each satellite involves rotation-translation coupling and nonlinearities. • Optimal formation control for the real uncertain coupled multi-satellite system can be achieved. • A two-step learning scheme is proposed to address the rotation-translation coupled dynamics. • The accurate dynamical model parameters are not required to be known by the learning method. [ABSTRACT FROM AUTHOR]

• Agents communicate indirectly and asynchronously via a shared cloud repository. • Both globally and locally Lipschitz nonlinearity in system dynamics are discussed. • Adaptive control scheme is considered for multi-agent systems under directed graph. • Practical tracking consensus is achieved for multi-agent systems with disturbances. • Zeno-freeness is guaranteed for the proposed cloud-based self-triggered scheme. This paper addresses the practical tracking consensus problem for first-orderheterogeneous nonlinear multi-agent systems in presence of external disturbances under directed graphs. Agents with limited communication capability interact with each other through a cloud repository, where the information exchange is asynchronous and indirect. The system model is more general than the ones in existing results, moreover, locally Lipschitz nonlinearities and adaptive control schemes are taken into account. It is proven that under the designed self-triggered protocol, agents can track a virtual leader with bounded error and Zeno behaviour is excluded for real implementation. Finally, a numerical simulation demonstrates the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

• This paper studies the leader-following flocking problem of the unmanned aerial vehicle (UAV). • It employs hybrid event-triggered control and pinning mechanism with delayed control inputs. • Criteria on flocking stability and Zeno behaviour prevention are derived. • It combines both displacement-based control and gyroscopic forces. • The tolerance of network instability in terms of continuous part is increased. This paper studies the leader-following flocking control problem of unmanned aerial vehicle (UAV) systems by integrating a hybrid impulsive framework, where the full exchange of information only occurs at each impulsive time instant. It is also shown that the overall stability can be achieved via impulses while the continuous dynamics remains destabilizing. Meanwhile, the even-triggered and pinning mechanisms are considered to further reduce control resource usage and transmission redundancy. Moreover, topology switching and strong nonlinearity are taken into account for more practical application. Based on transmission topology structure, impulsive control theory, and Lyapunov based event-triggered strategy, some improved theorems are derived to guarantee convergence of the corresponding error dynamics without exhibiting Zeno behaviour. Compared with most existing results on continuous or impulsive systems, the obtained stability criteria are more general as they are applicable to systems involving both delayed coupling feedback and delayed impulses. In addition, a novel approach using braking and gyroscopic forces is utilized for collision avoidance purposes. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical analysis. [ABSTRACT FROM AUTHOR]

Almost all systems are inevitably subject to various uncertainties or disturbances from the external environment in practical applications. Taking these factors into consideration, in this paper a distributed algorithm with state noise and gradient disturbance is proposed for solving distributed optimization problem with closed convex set constraint based on multi-agent system under weight-balanced graph. Moreover, based on the gradient tracking and projection methods, the proposed distributed algorithm with gradient tracking can improve the convergence rate by introducing a projection error term and an auxiliary parameter. In contrast to some existing constrained distributed gradient algorithms, the proposed one can make the convergence faster and enhance the performance of convergence. The proposed algorithm is illustrated with two simulation examples to show its effectiveness and robustness. [ABSTRACT FROM AUTHOR]

This paper investigates the optimal consensus problem for heterogeneous discrete-time(DT) linear multi-agent systems. The optimal consensus problem is formulated as finding a global Nash equilibrium solution subjected to the defined local performance index. A reinforcement learning(RL) value iteration(VI) algorithm is introduced to obtain the optimal policies in the sense of Nash equilibrium. To ensure the effectiveness of the VI algorithm, the admissibility of the iterative control policies for multi-agent systems is considered. With theoretical analysis, a new termination criterion is established to guarantee the admissibility of the iterative control policies. Furthermore, an online learning framework is designed with an actor-critic neural network(NN) to implement the VI algorithm. Finally, two simulation examples are presented respectively for leader–follower and leaderless multi-agent systems to verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this study, we successfully address the formation tracking challenge for underactuated autonomous surface vehicles (ASVs) in uncertain environments, despite the lack of direct target information. Leveraging only relative angle measurements, we develop a control scheme that ensures the closed-loop error trajectory remains globally uniformly ultimately bounded. The bound achieved is influenced by the precision of the dynamic controllers and the effectiveness of the adaptive mechanism, highlighting the robustness and adaptability of our approach. This work provides a practical solution for ASV formation tracking in real-world scenarios. • Introduces relative angle-based ASV guidance, mitigating uncertainty. • Achieves convergence with smaller, rational virtual signal. • Error trajectory GUUB, bound depends on controller precision. [ABSTRACT FROM AUTHOR]

For continuous-time linear leader-follower multi-agent systems with polynomial faults, the problem of fault-tolerant state consensus under switching topologies is studied. firstly, the average dwell time method is introduced, and observers are designed for different topologies to observe faults and disturbances. Secondly, according to the observed estimated value, the corresponding fault-tolerant control consensus protocol is proposed considering the switching situation of the communication topology. Finally, both theoretical analysis and simulation examples verify that the proposed design scheme can compensate faults and disturbance rejection. [ABSTRACT FROM AUTHOR]

This paper focuses on utilizing deep reinforcement learning technology to explore how to achieve automated architectural space composition under established built environment conditions to address the extensive demands of old building renovation. A reinforcement learning platform has been develop0065d, centered around the multi-agent deep deterministic policy gradient (MADDPG) algorithm. The paper leverages functional blocks as agents within Grasshopper to simulate existing architectural structures and their interactions, establishing a task-based reward system to guide the design process. Furthermore, data exchange between algorithmic and modeling software is facilitated through UDP communication. Training results using typical frame structure spaces indicate a significant increase in cumulative rewards around the 650,000th training step, effectively achieving the given design tasks. The paper demonstrates the potential of automated architectural space composition methods, based on deep reinforcement learning, to enhance design efficiency, optimize spatial arrangements, and promote human-machine collaborative design in building renovations. • A reinforcement learning platform using MADDPG for architectural renovation, with agents as functional blocks in a simulated environment. • A reward system optimizing spatial arrangements, considering function relationships and existing immovable structures. • Experimental validation shows increased cumulative rewards and effective task fulfillment in typical frame structure spaces. • Generation logic is interpretable, with editable vector outputs, controllable conditions, and promotes human-computer collaborative design. [ABSTRACT FROM AUTHOR]

In this paper, a smoothing approximation-based adaptive neurodynamic approach is proposed for a nonsmooth resource allocation problem (NRAP) with multiple constraints. The smoothing approximation method is combined with multi-agent systems to avoid the introduction of set-valued subgradient terms, thereby facilitating the practical implementation of the neurodynamic approach. In addition, using the adaptive penalty technique, private inequality constraints are processed, which eliminates the need for additional quantitative estimation of penalty parameters and significantly reduces the computational cost. Moreover, to reduce the impact of smoothing approximation on the convergence of the neurodynamic approach, time-varying control parameters are introduced. Due to the parallel computing characteristics of multi-agent systems, the neurodynamic approach proposed in this paper is completely distributed. Theoretical proof shows that the state solution of the neurodynamic approach converges to the optimal solution of NRAP. Finally, two application examples are used to validate the feasibility of the neurodynamic approach. [ABSTRACT FROM AUTHOR]

This paper studies an optimal synchronous control protocol design for nonlinear multi-agent systems under partially known dynamics and uncertain external disturbance. Under some mild assumptions, Hamilton–Jacobi–Isaacs equation is derived by the performance index function and system dynamics, which serves as an equivalent formulation. Distributed policy iteration adaptive dynamic programming is developed to obtain the numerical solution to the Hamilton–Jacobi–Isaacs equation. Three theoretical results are given about the proposed algorithm. First, the iterative variables is proved to converge to the solution to Hamilton–Jacobi–Isaacs equation. Second, the L 2 -gain performance of the closed loop system is achieved. As a special case, the origin of the nominal system is asymptotically stable. Third, the obtained control protocol constitutes an Nash equilibrium solution. Neural network-based implementation is designed following the main results. Finally, two numerical examples are provided to verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Centralized Training with Decentralized Execution (CTDE) is a prevalent paradigm in the field of fully cooperative Multi-Agent Reinforcement Learning (MARL). Existing algorithms often encounter two major problems: independent strategies tend to underestimate the potential value of actions, leading to the convergence on sub-optimal Nash Equilibria (NE); some communication paradigms introduce added complexity to the learning process, complicating the focus on the essential elements of the messages. To address these challenges, we propose a novel method called O ptimistic S equential S oft Actor Critic with M otivational C ommunication (OSSMC). The key idea of OSSMC is to utilize a greedy-driven approach to explore the potential value of individual policies, named optimistic Q-values, which serve as an upper bound for the Q-value of the current policy. We then integrate a sequential update mechanism with optimistic Q-value for agents, aiming to ensure monotonic improvement in the joint policy optimization process. Moreover, we establish motivational communication modules for each agent to disseminate motivational messages to promote cooperative behaviors. Finally, we employ a value regularization strategy from the Soft Actor Critic (SAC) method to maximize entropy and improve exploration capabilities. The performance of OSSMC was rigorously evaluated against a series of challenging benchmark sets. Empirical results demonstrate that OSSMC not only surpasses current baseline algorithms but also exhibits a more rapid convergence rate. • We propose an optimistic Q-value to explore the potential value of individual policies. • We introduce specially designed motivational communication modules to enhance teamwork. • We show that OSSMC can achieve a competitive baseline in many challenging benchmarks. [ABSTRACT FROM AUTHOR]

Flocking cohesion is a crucial factor for groups to maintain aggregation. Communication barriers can significantly challenge the maintenance of flocking aggregation and integrity. Although several studies have focused on communication barriers between agents and the Leader, less attention has been given to agents' self-communication barriers. This paper investigates the effect of two kinds of communication barriers that coexist on flocking cohesion. One is the communication barrier between agents and the Leader. There are informed and uninformed agents, with informed agents receiving different degrees of complete information about the Leader. The other is the agent's self-communication barrier, where each agent has a failure rate that prevents it from obtaining and transmitting information. Then, a novel potential function is designed by analyzing the correlation between the potential function and flocking cohesion. It enhances flocking cohesion and reduces flocking time. Moreover, the flocking control algorithm is improved by incorporating a local feedback mechanism. It allows informed agents to transfer information more broadly and promotes interaction among agents, further enhancing flocking cohesion and integrity. Stability analysis is performed using the Lyapunov theorem. Finally, we propose three criteria for evaluating flocking performance. Based on these criteria, simulation results demonstrate the proposed potential function and control algorithm's significant advantages. This control algorithm is applicable for addressing individual communication issues in swarming drones or flocking robots during missions in engineering applications. [ABSTRACT FROM AUTHOR]

How do we know when a reinforcement learning policy needs to adapt? In non-stationary environments, agents must adapt and learn in environments that change dynamically. We propose a finite-horizon model-free solution using a hierarchical learning structure with fuzzy systems. The higher-level learning policy advises the lower-level policy when to start and stop learning based on the temporal differences calculated within the lower-level. Major differences in the temporal difference of each action produced by an agent may indicate environment change. This structure is tested with multi-agent differential games in both the cooperative and competitive aspect. Results show that this method is quick to notice and adapt the policy within relatively few learning episodes. [ABSTRACT FROM AUTHOR]

There are fewer simulation studies that comprehensively consider the impact of collision events due to heterogeneous driving behaviour on multi-lane traffic flow. This comprehensive study utilized multi-agent modeling to simulate the driver-vehicle-road system at the mesoscale level, integrating the characteristics of heterogeneous driving behavior. We developed a traffic flow simulation model for a three-lane urban arterial road segment and used rule-based algorithms to simulate vehicle following, lane changing, and collision behavior. A visualization simulation program was developed using NetLogo software to observe and analyze vehicle behavior. The findings revealed that collision events were the leading cause of traffic bottlenecks, impacting the continuity and coherence of traffic flow. Even after the collision event subsided and congestion eased, the average vehicle speed could not fully recover to its original level, reaching only about 70% of the maximum vehicle speed. Aggressive drivers exhibited distinct speed control strategies compared to conservative and ordinary drivers. This study demonstrates the effectiveness of multi-agent modeling in capturing the relationship between traffic bottlenecks and collision events, highlighting the influence of dynamic traffic events on driving behavior. [ABSTRACT FROM AUTHOR]

In the realm of fully cooperative multi-agent reinforcement learning (MARL), effective communication can induce implicit cooperation among agents and improve overall performance. In current communication strategies, agents are allowed to exchange local observations or latent embeddings, which can augment individual local policy inputs and mitigate uncertainty in local decision-making processes. Unfortunately, in previous communication schemes, agents may potentially receive irrelevant information, which increases training difficulty and leads to poor performance in complex settings. Furthermore, most existing works lack the consideration of the impact of small coalitions formed by agents in the multi-agent system. To address these challenges, we propose HyperComm , a novel framework that uses the hypergraph to model the multi-agent system, improving the accuracy and specificity of communication among agents. Our approach brings the concept of hypergraph for the first time in multi-agent communication for MARL. Within this framework, each agent can communicate more effectively with other agents within the same hyperedge, leading to better cooperation in environments with multiple agents. Compared to those state-of-the-art communication-based approaches, HyperComm demonstrates remarkable performance in scenarios involving a large number of agents. [Display omitted] • A novel MARL communication protocol named HyperComm is proposed. • The hypergraph is utilized in multi-agent communication for the first time. • A two-stage information broadcasting approach is designed. • The performance of HyperComm is evaluated on diverse MARL benchmarks. [ABSTRACT FROM AUTHOR]

It is well known that nonlinear multi-agent systems (MASs) with a directed topology can reach a consensus if the global coupling coefficient is strong enough. In this paper, we propose a novel distributed protocol based on the subgroup structure. Our results demonstrate that the MAS design can follow a part-to-whole approach, starting with subgroups and gradually building up to a more complex system. We firstly consider two cases for the MASs: one with a single leader and one with a leader group, where agents within the leader group can communicate with each other, but cannot receive the information of other group. We show that followers can track the leader or group leaders if intra-coupling within the subgroup and information strength received by the root node are above a certain threshold. Next, we investigate cluster consensus and practical cluster consensus for the Laplacian matrix with unequal row sum, equal row sum, and mixing of partitioned matrices. Our results reveal that row sum and community structure of the partitioned matrix are crucial in determining the final state of each agent. Our proposed control protocol is highly flexible and relies only on the cluster structure and information received by the root node, rather than global information. Moreover, our method is not limited to tracking problems, cooperative control, or multi-objective control, and can accommodate new group agents joining the system, making it highly adaptable. We provide illustrative examples to demonstrate the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Agent negotiation in multi-agent systems has been extensively studied, focusing on both theoretical and applied research. However, a limited number of studies have considered proposing an offer-generating strategy for agents to propose offers during the negotiation process in the multiple-negotiation situation where interdependency exist between a mixture of discrete issues and continuous issues across different negotiations. Especially, considering the above common real-life situation, there is little work of proposing such a strategy which is able to generate an approximately Pareto optimal solution. To address such challenges, this paper targets at multiple-negotiation scenarios involving interdependency between mixed types of issues across different negotiations. The contributions of this paper are threefold. Firstly, this paper addresses the research gap in mixed-type of issues in multiple negotiations. Secondly, the paper introduces a formalized negotiation model for multiple-negotiation scenarios, addressing both discrete and continuous issues, enabling automatic agents to obtain goal-aligned offers effectively. Thirdly, this paper introduces a Hybrid of PSO (Particle Swarm Optimization) and GA (Genetic Algorithm) Algorithm (i.e., named as HPGA in this paper) as an offer-generating strategy to assist agents in achieving approximately Pareto optimization in multiple-negotiation scenarios. To support those claims, this paper presents an overall modeling framework, introduces the proposed offer-generation strategy, conducts a series of experiments to demonstrate the superiority of the proposed approach in this paper, and presents two realistic case studies. Overall, this research expands upon existing studies in agent-based negotiation by addressing the overlooked aspects of mixed types of issues and issue interdependency across multiple negotiations. The proposed modeling approach and offer-generation strategy contribute to the advancement of negotiation techniques in multi-agent systems. • Fill the research gap of multi-negotiations with mixed-types of issues. • Propose a unified modeling for multi-issue negotiations scenario. • Propose an algorithm named HPGA as the offer-generating strategy. • Help agents to achieve approximately Pareto optimal by applying the proposed method. [ABSTRACT FROM AUTHOR]

In recent years, in the field of Multi-Agent Systems (MAS), significant progress has been made in the research of algorithms that combine Reinforcement Learning (RL) with Attention Mechanism (AM). However, there is a lack of comprehensive reviews in this field. Based on this, this paper does the following work. Firstly, it reviews the classical algorithms of RL and AM; Secondly, it systematically introduces the combination of RL and AM; Thirdly, it sorts out their application in the field of single-agent and multi-agent, and pays attention to and looks forward to the challenges and future research prospects in this field; The last part offers a comprehensive analysis of the challenges encountered by research in this field and anticipates future research paths. The research offers a conceptual understanding and theoretical foundation for future applications of RL using AM and facilitates further in-depth study in this field for researchers. [ABSTRACT FROM AUTHOR]

This paper considers the exponential consensus of nonlinear switched multi-agent systems (MASs) subject to random attacks. The proposed model is enhanced by introducing the concept of mode-dependent average dwell time (MDADT) and the one-sided Lipschitz (OSL) condition, making it more practical and versatile with relaxed restrictions. Due to the unavailability of direct access to the states of MASs and the vulnerability of their output signals to unknown random attacks, achieving consensus through traditional methods becomes challenging. Hence, a novel attack-decomposition approach is proposed to extract the attack-free output signals from the attacked output signals. Then, the observer is constructed by utilizing attack-free output signals and employed to estimate the states of MASs. A mode-dependent event-triggering mechanism is presented to conserve resources for the observer-controller channels. In order to save control costs, the intermittent control scheme is designed for different switching modes whose control width depends on the minimum dwell time. Finally, an illustrative example is used to showcase the advantages of the theoretical method. [ABSTRACT FROM AUTHOR]

This paper is concerned with adaptive time-varying formation tracking problems of multi-agent systems (MASs) with uncertain communication networks, non-parametric actuator faults and matched and mismatched uncertainties/disturbances. Considering the leader–follower structure of the MAS and the complex working environment, the communication networks are unstationary, for which, a double Markov process is first introduced to characterize this topology. A stochastic sliding surface is designed for each follower, and the stability of the corresponding sliding mode dynamics is analyzed. An adaptive sliding mode controller is developed to drive the dynamical system to the sliding surface in finite time and maintains a sliding motion thereafter. The stability of the corresponding sliding mode and the reachability are proved via Lyapunov direct method. Finally, a simulation example of three unmanned aerial vehicles (UAVs) and one unmanned ground vehicle (UGV) is given to demonstrate the effectiveness of the proposed strategy. [ABSTRACT FROM AUTHOR]

In this paper, a distributed control algorithm based on a data-driven approach is developed to solve the consensus control of heterogeneous nonlinear Multi-Agent Systems (MAS). The consensus obtained from the solution of the Hamilton–Jacobi–Bellman (HJB) equation is challenging for unknown nonlinear systems. To address this issue, improved online reinforcement learning (RL) is employed to generate an approximate solution for each agent to achieve consensus. Unlike model-based RL and traditional algorithms, this method leverages I/O data to guide the learning of policies without any prior knowledge of agent dynamics. Furthermore, the adaptability of algorithm to heterogeneous nonlinear agents is enhanced by implementing online updates to the control strategy and dynamic linearization (DL). The convergence analysis of the algorithm is provided, along with the impact of the learning rate parameters on the consensus of MAS. By comparing with other data-driven methods, simulations are conducted to verify the stability and adaptability of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In the context of the automotive sector, production processes are characterized by an extensive variety of products, high and fluctuating production volume and numerous vehicle components. These features make internal logistics vastly challenging and prone to perturbations. Traditional control systems for manufacturing processes are evolving to more modular, intelligent and distributed systems. The existing literature reports that Multi-Agent Systems (MAS) is a suitable approach to implement these new control systems. However, the number of reported real world implementations in the industrial field is limited. In addition, no study has been found that considers the application of MAS techniques in the internal logistics of an industrial process. This article presents a MAS-based methodology for reducing the reaction time to a part missing alert triggered by an operator in an assembly line that uses the kitting process. A contract net Protocol was adapted to manage the interaction between the involved agents. With the purpose of implementing the proposed methodology on the industrial field, operators play an important role in the decision-making phase. The proposed MAS-based methodology was tested in a real-case scenario from a Renault Group plant. Historical data collected from the assembly line is used to model the system at the time in which an operator triggered the alert. The MAS-based approach was applied to propose solutions to address this perturbation. The obtained results demonstrate the effectiveness of the proposed approach by providing solutions that reduce the reaction time in more than 60%, compared to the production plant current approach. Future work will be devoted to the implementation of the proposed methodology on the industrial field. [ABSTRACT FROM AUTHOR]

Traffic guidance and traffic control are effective means to alleviate traffic problems. Formulating effective traffic guidance measures can improve the utilization rate of road resources and optimize the performance of the entire traffic network. Assuming that the traffic coordinator can capture traffic flow information in real-time utilizing sensors installed on each road, we consider the strong resilience constraints to construct an optimal selection problem of balanced flow in the traffic network. Based on multi-agent modeling, each agent has access to the corresponding traffic information of each link. We design a distributed optimization algorithm to tackle this optimization problem. In addition to the inherent advantages of distributed multi-agent algorithms, the communication topology among the sensors is allowed to be time-varying, which is more consistent with reality. To prove the effectiveness of the proposed algorithm, we also give a numerical simulation in the multi-agent environment. It should be reiterated that the optimization problem studied in this paper mainly focuses on traffic managers' perspectives. The goal of the studied optimization problem is to minimize the overall cost of the traffic network by adjusting the optimal equilibrium traffic flow. This study provides a reference for solving congestion optimization in today's cities. • Formulate an equilibrium flow selection problem with resilience constraints. • The distributed algorithm allows the communication among agents to be time-varying. • Communication graph among agents allows for being B-connected. [ABSTRACT FROM AUTHOR]

Abnormal combustion is an important factor in the development process of hydrogen engine and it mainly includes pre-ignition, backfire and knocking, among which pre-ignition has the most serious impact on hydrogen engine. In this paper, it is divided into four types: normal combustion, slight pre-ignition, moderate pre-ignition and severe pre-ignition according to different crankshaft rotation angles. In order to identify different combustion types, this paper proposes a fault diagnosis model based on the fusion of SOM neural network and Multi-Agent System (SOM-MAS). Firstly, different combustion types are identified by SOM. Secondly, the abnormal combustion is tracked and located mainly through the Multi-Agent System, and the location of the abnormality is identified. Finally, based on 44 sets of pressure data samples collected from the in-cylinder combustion of a hydrogen engine on the experimental bench, different combustion types were diagnosed and identified, and the location of abnormal combustion faults was tracked, which verifies the effectiveness of the proposed method shows that the method has certain feasibility and superiority for the diagnosis of hydrogen engine pre-ignition. • Research carried out by the 492Q hydrogen engine, close to actual conditions. • The dataset is original and contains data for normal and abnormal combustion. • SOM-MAS can be applied in abnormal combustion of hydrogen fuel engines. • Less relevant dataset, we will publish the dataset for future research. [ABSTRACT FROM AUTHOR]

This paper considers the event-triggered predictive control(ETPC) problem for linear discrete-time multi-agent systems. To achieve accurate state estimation under event-triggered communication between agents, an event-triggered predictive control scheme as well as a novel class of triggering condition are proposed for synchronization of multi-agent systems with single-integrator dynamics, where the control input of each agent no longer remains constant but updates periodically within the time interval between two consecutive triggering instants. Then, the proposed scheme is extended to the general linear multi-agent systems. By comparing with the time-triggered control(TTC) method and the existing event-triggered control(ETC) methods, the control performance of the proposed ETPC scheme is almost the same as that of the TTC approach, and the communication payload between agents is reduced with much less triggering times than existing ETC methods. Finally, two numerical examples are given to verify the theoretical analysis and demonstrate the superiority of the proposed method comparing with some existing methods. [ABSTRACT FROM AUTHOR]

The MS-RCPSP belongs to the class of the strongly NP-hard optimisation problems. The MS-RCPSP is an extension of the classical RCPSP where some given pool of skills has been assigned to the resources. To solve this problem the multi-agent system approach has been proposed, implemented and used. The A-Team multi-agent system has been built using the environment where optimisation agents are used to find solutions. The approach has been tested experimentally using benchmark problem instances from iMOPSE dataset with the makespan optimization criterion [ABSTRACT FROM AUTHOR]

The modern world is based on information. With the current development of communication technologies, messages are spreading rapidly through social networks, gathering vast numbers of users. Along with facilitating the transfer of information on a large scale, more remarkable ability to apply and influence life, politics and money have so-called "fake news", understood as artificially crafted false information to achieve specific goals. The growing influence of such fake news increases interest in social scientists and generates the need to develop measures to reduce this phenomenon. Implementing countermeasures on a large scale is costly, and the results are often uncertain. Therefore, models and systems are being developed to predict the dynamics and course of the spread of "fake news" to study the influence of various factors on this very process. This article presents a multi-agent simulation system for spreading fake information in the so-called scale-free networks. [ABSTRACT FROM AUTHOR]

In this paper we propose a solution to a challenging task of magnetic resonance image reconstruction in a scalable and flexible way. The proposal elaborates task slicing and its distribution using market-based approach, where participating nodes perform task processing using their own spare computing capacities. The distributed task execution is orchestrated within multi-agent environment where agents offer the execution services by bidding for the offered tasks. Proposed negotiation protocol includes local and cloud-based resources where the offered services are exchanged for an award based on the negotiated award value function. The proposed solution considers simplicity and expressiveness of SDLMAS language and the advantages of corresponding platform for rapid design and development of multi-agent systems. [ABSTRACT FROM AUTHOR]

Now more than ever, optimizing resource allocation in cloud computing is becoming more critical due to the growth of cloud computing consumers and meeting the computing demands of modern technology. Cloud infrastructures typically consist of heterogeneous servers, hosting multiple virtual machines with potentially different specifications, and volatile resource usage. This makes the resource allocation face many issues such as energy conservation, fault tolerance, workload balancing, etc. Finding a comprehensive solution that considers all these issues is one of the essential concerns of cloud service providers. This paper presents a new resource allocation model based on an intelligent multi-agent system and reinforcement learning method (IMARM). It combines the multi-agent characteristics and the Q-learning process to improve the performance of cloud resource allocation. IMARM uses the properties of multi-agent systems to dynamically allocate and release resources, thus responding well to changing consumer demands. Meanwhile, the reinforcement learning policy makes virtual machines move to the best state according to the current state environment. Also, we study the impact of IMARM on execution time. The experimental results showed that our proposed solution performs better than other comparable algorithms regarding energy consumption and fault tolerance, with reasonable load balancing and respectful execution time. [ABSTRACT FROM AUTHOR]

Sustainable energy resource implementation has several advantages in terms of energy efficiency, reliability, and resilience. However, there are still challenges with the power quality in suitable energy balance and acceptable voltage levels in the electrical network. Therefore, this study presents novel energy coordination for implementing grid-tied microgrids, including photovoltaic and battery energy storage systems. Thus, multi-agent modelling based on system analysis is implemented to formulate the dynamic performance of independent, interconnected and autonomous microgrids. Three optimal control schemes, including open-loop, closed-loop and model predictive control, are combined with the optimal power flow algorithm to dynamically coordinate each independent agent and the entire microgrid. The system results demonstrate that, by combining these strategies with several smart homes, the dynamic coordination of microgrids brings various benefits, such as important economic, environmental and operation performance indicators, voltage stability, power loss minimisation and power-saving. The validation of the designed approaches is bench-marked within a 33-bus IEEE network in the residential sector. The developed intelligent coordination structures achieve significant energy savings ranging from 23.99% to 36.14% while maintaining suitable system voltage levels and minimising power loss. Besides, the developed dynamic coordination offers an adequate scalability framework for an effective microgrid implementation. [Display omitted] • Implement a real-world demand management model for residential applications. • Formulate tertiary control strategies based on EMS and MAS for optimal coordination of microgrids. • Coordinate distributed energy resources, including solar PV and energy storage tied to the grid. • Employ a dynamic approach to ensure efficient utilisation of available energy resources within smart microgrids. • Provide a microgrid KPI framework based on techno-economic and environmental assessment. [ABSTRACT FROM AUTHOR]

Scheduling in decentralized manufacturing systems is being expanded and transformed to a smart level where manufacturing tasks are executed autonomously. However, achieving self-organized manufacturing resources that can make collaborative decisions remains challenging. This paper presents a novel negotiation model for decentralized scheduling in an autonomous manufacturing system. A variety of emotions represent the willingness of autonomous intelligent agents (IAs) to negotiate for a consensus despite their diverse goals. The proposed communication protocol considers the emotional aspects of the autonomous machine agents in decentralized decision-making. The negotiation model focuses on analyzing the effects of two key emotions: selfishness and cooperation. This study constitutes the basis for experiments on multi-goal machine tasks and artificial emotion mechanism in a flow shop production system. The complexities of negotiation dynamics between IAs, where the interplay of selfish and cooperative strategies influences the outcomes and efficiency of reaching agreements, are explored. The experimental results show that selfish IAs eventually reach agreements through gradual compromise, but cooperative IAs, with their flexibility, can quickly achieve resolutions, especially when bargaining power is balanced. Aligning with the dual concern theory, this study reveals that while selfish IAs can dominate cooperative ones, the cooperative approach often leads to faster and more effective outcomes, regardless of the level of selfishness involved. [ABSTRACT FROM AUTHOR]

Improving the efficiency of home energy management (HEM) is of great significance in reducing resource waste and prompting renewable energy consumption. With the development of advanced HEM systems and internet-of-things technology, more residents are willing to participate in the electricity market through a demand response mechanism, which can not only reduce the electricity bill but also stabilize the operation of the main grid through peak shaving and valley filling. However, the uncertainties from household appliance parameters, user behaviors, penetrated renewable energy, and electricity prices render the HEM problem a non-trivial task. Although previous deep reinforcement learning (DRL) methods have no requirement for system dynamics due to their model-free and data-driven merits, the unrestricted action space may violate the physical constraints of various devices, and few works have concentrated on this area before. To solve the above challenges, this paper proposes a novel interior-point policy optimization-based multi-agent DRL algorithm to optimize the home energy scheduling procedure while guaranteeing safety during its operation. Besides, a time-series prediction model based on a non-stationary Transformer neural network is proposed to predict the future trend of solar generation and electricity price, which can provide the agents with abundant information to make better decisions. The superior performance of our proposed predictive-control coupled method is demonstrated through extensive numerical experiments, which achieves more precise prediction results, near-zero constraint violations, and better trade-offs between cost and safety than several benchmark methods. • A data-driven secure home energy management model under various uncertainties is proposed. • Multi-agent deep reinforcement learning framework based on deep deterministic policy gradient is established. • An interior-point optimization-based secure decision-making approach to conduct the home energy management process. • The uncertainties from photovoltaic and electricity price are solved by utilizing a novel non-stationary Transformer deep model. [ABSTRACT FROM AUTHOR]

This paper is concerned with the problem of dynamic event-triggered bipartite consensus control for nonlinear multi-agent systems based on signed networks. A new dynamic event-triggered control mechanism is proposed, whereby an adjustment variable is introduced to dynamically schedule the triggering frequency, enabling the minimization of the triggering times while ensuring system performance. Based on the designed event-triggered control algorithm, sufficient conditions are derived to guarantee that bipartite consensus can be reached by the considered nonlinear multi-agent system. Furthermore, it is proved that Zeno behavior will not occur. Numerical examples are provided in this paper to verify the effectiveness of the proposed control algorithm. The simulation results reveal that, compared with the static event-triggered mechanism and the traditional dynamic event-triggered mechanism, the proposed event-triggered algorithm can further reduce the triggering times and enhance the flexibility of the event-triggered mechanism. [ABSTRACT FROM AUTHOR]

Multi-agent reinforcement learning requires numerous interactions with the environment and other agents to learn an optimal policy. The teacher–student framework is one paradigm that can enhance the learning performance of reinforcement learning by allowing agents to seek advice from one another. However, recent studies show limitations in knowledge sharing between agents, as advisory learning is only peer-to-peer at a given time. Some methods enable a student to accept multiple pieces of advice, but they typically rely on pre-trained and/or policy-fixed teachers, rendering them unsuitable for agents with simultaneous advisory learning. Simultaneous learning with multiple pieces of advice has not been thoroughly investigated. Furthermore, most research has concentrated on the sharing of knowledge samples, a practice vulnerable to security breaches that could allow attackers to deduce details about the environment. To address these challenges, we propose a federated advisory framework that uses a federated learning structure to aggregate multiple sources of advice with deep reinforcement learning, ensuring that the shared advice is not sample-based. Our experimental comparisons with leading advisory learning techniques confirm that our approach significantly enhances learning performance. [ABSTRACT FROM AUTHOR]

This paper considers consensus strategy design for double-integrator multi-agent systems with matched disturbances under a directed graph. Specifically, we consider the case where both the control inputs and velocities of the agents are subject to different and asymmetric constraints. A novel distributed algorithm exploiting saturation functions is proposed to achieve asymptotic consensus without violating the predefined velocity and actuator saturation constraints. Subsequently, we extend the obtained results to event-triggered communication, where agent position broadcasting occurs only when specific triggering conditions are met. We rigorously prove that there exists a positive minimum inter-event time to rule out Zeno behavior. Finally, the simulation results confirm the theoretical findings and illustrate the achievable performance. [ABSTRACT FROM AUTHOR]