Search

The duration of the accumulation rate (physical image) is a key factor in analysis of counterintuitive phenomena involving relative timescales on multiplex networks. Typically, the relative timescales are represented by multiplying any layer by the same factor. However, researchers often overlook the changes in the relative timescales caused by local parameters, resulting in incomplete analysis of phenomena. This paper examines the survival time of stifler individuals in the information-epidemic model on multiplex networks. The relative timescales can be affected by the survival time (only one parameter), reversing the monotonically increasing phenomenon into a monotonically decreasing one, that is, a counterintuitive phenomenon under incomplete analysis. Additionally, the relative timescales can influence the epidemic threshold, which is different from the previous studies. Our work suggests that considering the physical image of relative timescales is crucial when analyzing multiplex networks, even when only one parameter is altered.

Patterns by self-organization in nature have garnered significant interest in a range of disciplines due to their intriguing structures. In the context of the snowdrift game (SDG), which is considered as an anti-coordination game, but the anti-coordination patterns are counterintuitively rare. In the work, we introduce a model called the Two-Agents, Two-Action Reinforcement Learning Evolutionary Game ($2\times 2$ RLEG), and apply it to the SDG on regular lattices. We uncover intriguing phenomena in the form of Anti-Coordinated domains (AC-domains), where different frustration regions are observed and continuous phase transitions at the boundaries are identified. To understand the underlying mechanism, we develop a perturbation theory to analyze the stability of different AC-domains. Our theory accurately partitions the parameter space into non-anti-coordinated, anti-coordinated, and mixed areas, and captures their dependence on the learning parameters. Lastly, abnormal scenarios with a large learning rate and a large discount factor that deviate from the theory are investigated by examining the growth and nucleation of AC-domains. Our work provides insights into the emergence of spatial patterns in nature, and contributes to the development of theory for analysing their structural complexities.

Cooperation is the foundation of ecosystems and the human society, and the reinforcement learning provides crucial insight into the mechanism for its emergence. However, most previous work has mostly focused on the self-organization at the population level, the fundamental dynamics at the individual level remains unclear. Here, we investigate the evolution of cooperation in a two-agent system, where each agent pursues optimal policies according to the classical Q-learning algorithm in playing the strict prisoner's dilemma. We reveal that a strong memory and long-sighted expectation yield the emergence of Coordinated Optimal Policies (COPs), where both agents act like Win-Stay, Lose-Shift (WSLS) to maintain a high level of cooperation. Otherwise, players become tolerant toward their co-player's defection and the cooperation loses stability in the end where the policy all Defection (All-D) prevails. This suggests that tolerance could be a good precursor to a crisis in cooperation. Furthermore, our analysis shows that the Coordinated Optimal Modes (COMs) for different COPs gradually lose stability as memory weakens and expectation for the future decreases, where agents fail to predict co-player's action in games and defection dominates. As a result, we give the constraint to expectations of future and memory strength for maintaining cooperation. In contrast to the previous work, the impact of exploration on cooperation is found not be consistent, but depends on composition of COMs. By clarifying these fundamental issues in this two-player system, we hope that our work could be helpful for understanding the emergence and stability of cooperation in more complex scenarios in reality., Comment: 29 pages,11 figures

Large-scale cooperation underpins the evolution of ecosystems and the human society, and the collective behaviors by self-organization of multi-agent systems are the key for understanding. As artificial intelligence (AI) prevails in almost all branches of science, it would be of great interest to see what new insights of collective behavior could be obtained from a multi-agent AI system. Here, we introduce a typical reinforcement learning (RL) algorithm -- Q learning into evolutionary game dynamics, where agents pursue optimal action on the basis of the introspectiveness rather than the birth-death or imitation processes in the traditional evolutionary game (EG). We investigate the cooperation prevalence numerically for a general $2\times 2$ game setting. We find that the cooperation prevalence in the multi-agent AI is amazing of an equal level as in the traditional EG in most cases. However, in the snowdrift games with RL we also reveal that explosive cooperation appears in the form of periodic oscillation, and we study the impact of the payoff structure on its emergence. Finally, we show that the periodic oscillation can also be observed in some other EGs with the RL algorithm, such as the rock-paper-scissors game. Our results offer a reference point to understand emergence of cooperation and oscillatory behaviors in nature and society from AI's perspective., Comment: 10 pages, 7 figures

Resource allocation takes place in various types of real-world complex systems such as urban traf- fic, social services institutions, economical and ecosystems. Mathematically, the dynamical process of complex resource allocation can be modeled as minority games in which the number of resources is limited and agents tend to choose the less used resource based on available information. Spontaneous evolution of the resource allocation dynamics, however, often leads to a harmful herding behavior accompanied by strong fluctuations in which a large majority of agents crowd temporarily for a few resources, leaving many others unused. Developing effective control strategies to suppress and elim- inate herding is an important but open problem. Here we develop a pinning control method. That the fluctuations of the system consist of intrinsic and systematic components allows us to design a control scheme with separated control variables. A striking finding is the universal existence of an optimal pinning fraction to minimize the variance of the system, regardless of the pinning patterns and the network topology. We carry out a detailed theoretical analysis to understand the emergence of optimal pinning and to predict the dependence of the optimal pinning fraction on the network topol- ogy. Our theory is generally applicable to systems with heterogeneous resource capacities as well as varying control and network topological parameters such as the average degree and the degree dis- tribution exponent. Our work represents a general framework to deal with the broader problem of controlling collective dynamics in complex systems with potential applications in social, economical and political systems., Comment: 18 pages, 8 figures

Resource allocation takes place in various kinds of real-world complex systems, such as the traffic systems, social services institutions or organizations, or even the ecosystems. The fundamental principle underlying complex resource-allocation dynamics is Boolean interactions associated with minority games, as resources are generally limited and agents tend to choose the least used resource based on available information. A common but harmful dynamical behavior in resource-allocation systems is herding, where there are time intervals during which a large majority of the agents compete for a few resources, leaving many other resources unused. Ac- companying the herd behavior is thus strong fluctuations with time in the number of resources being used. In this paper, we articulate and establish that an intuitive control strategy, namely pinning control, is effective at harnessing the herding dynamics. In particular, by fixing the choices of resources for a few agents while leaving majority of the agents free, herding can be eliminated completely. Our investigation is systematic in that we consider random and targeted pinning and a variety of network topologies, and we carry out a comprehensive analysis in the framework of mean-field theory to understand the working of control. The basic philosophy is then that, when a few agents waive their freedom to choose resources by receiving sufficient incentives, majority of the agents benefit in that they will make fair, efficient, and effective use of the available resources. Our work represents a basic and general framework to address the fundamental issue of fluctuations in complex dynamical systems with significant applications to social, economical and political systems., Comment: 9 pages,4 figures

TheMinority Game (MG) has become a paradigm to probe complex social and economical phenomena where adaptive agents compete for a limited resource, and it finds applications in statistical and nonlinear physics as well. In the traditional MG model, agents are assumed to have access to global information about the past history of the underlying system, and they react by choosing one of the two available options associated with a single resource. Complex systems arising in a modern society, however, can possess many resources so that the number of available strategies/resources can be multiple. We propose a class of models to investigate MG dynamics with multiple strategies. In particular, in such a system, at any time an agent can either choose a minority strategy (say with probability p) based on available local information or simply choose a strategy randomly (with probability 1 - p). The parameter p thus defines the minority-preference probability, which is key to the dynamics of the underlying system. A striking finding is the emergence of strategy-grouping states where a particular number of agents choose a particular subset of strategies. We develop an analytic theory based on the mean-field framework to understand the "bifurcation" to the grouping states and their evolution. The grouping phenomenon has also been revealed in a real-world example of the subsystem of 27 stocks in the Shanghai Stock Market's Steel Plate. Our work demonstrates that complex systems following the MG rules can spontaneously self-organize themselves into certain divided states, and our model represents a basic mathematical framework to address this kind of phenomena in social, economical, and even political systems., Comment: 10 pages,8 figures and a real word example