Search

Patterns emerging through self-organization in nature have sparked considerable interest across various disciplines, owing to their significance in comprehending collective behaviors. The anti-coordination phenomenon is widespread in both social systems and ecosystems. However, within the context of the SnowDrift Game (SDG), considered as a typical anti-coordination game, the occurrence of anti-coordination patterns is surprisingly rare. In this work, we introduce a model called the Two-Agent, Two-Action Reinforcement Learning Evolutionary Game (2 × 2 RLEG), and utilize it in the SDG on different regular lattices. We unveil intriguing phenomena represented by Anti-Coordinated domains (AC-domains) within spatial patterns, which hold a significant sway over the cooperation preference within populations. The frustration contributes substantially to the diversity of AC-domains within parameter space. To comprehend the underlying mechanism, we have developed a perturbation theory to analyze the stability of different AC-domains comprising action patterns and identify continuous phase transitions within parameter space. Our theory effectively partitions the parameter space into several areas, each characterized by unique features of domains that determine the cooperation preferences within the population. Our work offers insights into the emergence of spatial patterns and their impact on cooperation in nature, contributing to the development of theories that analyze their structural complexities. • Anti-coordinated patterns naturally emerge in snowdrift game by reinforcement learning. • A perturbation theory is successfully developed. • The effect of memory and frustration greatly influence these patterns and collective preferences. [ABSTRACT FROM AUTHOR]

Cooperation is the foundation of ecosystems and the human society, and the reinforcement learning provides crucial insight into the mechanism for its emergence. However, most previous work has mostly focused on the self-organization at the population level, the fundamental dynamics at the individual level remains unclear. Here, we investigate the evolution of cooperation in a two-agent system, where each agent pursues optimal policies according to the classical Q-learning algorithm in playing the strict prisoner's dilemma. We reveal that a strong memory and long-sighted expectation yield the emergence of Coordinated Optimal Policies (COPs), where both agents act like "Win-Stay, Lose-Shift" (WSLS) to maintain a high level of cooperation. Otherwise, players become tolerant toward their co-player's defection and the cooperation loses stability in the end where the policy "all Defection" (All-D) prevails. This suggests that tolerance could be a good precursor to a crisis in cooperation. Furthermore, our analysis shows that the Coordinated Optimal Modes (COMs) for different COPs gradually lose stability as memory weakens and expectation for the future decreases, where agents fail to predict co-player's action in games and defection dominates. As a result, we give the constraint to expectations of future and memory strength for maintaining cooperation. In contrast to the previous work, the impact of exploration on cooperation is found not be consistent, but depends on composition of COMs. By clarifying these fundamental issues in this two-player system, we hope that our work could be helpful for understanding the emergence and stability of cooperation in more complex scenarios in reality. • Strong memory and long-sighted expectations yield "win-stay, lose-shift" and high cooperation. • Tolerance of exploitation could be a good precursor to a crisis in cooperation. • The impact of exploration on cooperation nonmonotonically depends on the composition of the coordinated optimal modes. [ABSTRACT FROM AUTHOR]