Search

In efforts to resolve social dilemmas, reinforcement learning is an alternative to imitation and exploration in evolutionary game theory. While imitation and exploration rely on the performance of neighbors, in reinforcement learning individuals alter their strategies based on their own performance in the past. For example, according to the Bush–Mosteller model of reinforcement learning, an individual's strategy choice is driven by whether the received payoff satisfies a preset aspiration or not. Stimuli also play a key role in reinforcement learning in that they can determine whether a strategy should be kept or not. Here we use the Monte Carlo method to study pattern formation and phase transitions towards cooperation in social dilemmas that are driven by reinforcement learning. We distinguish local and global players according to the source of the stimulus they experience. While global players receive their stimuli from the whole neighborhood, local players focus solely on individual performance. We show that global players play a decisive role in ensuring cooperation, while local players fail in this regard, although both types of players show properties of 'moody cooperators'. In particular, global players evoke stronger conditional cooperation in their neighborhoods based on direct reciprocity, which is rooted in the emerging spatial patterns and stronger interfaces around cooperative clusters. [ABSTRACT FROM AUTHOR]

• This paper introduces expectation-based reinforcement learning methods in the public goods game. • The results under total population density show that individuals behavior exhibits conditional cooperation and its variant moody conditional cooperation. • In the network with empty nodes, conditional cooperation shows opposite trends. In social dilemmas, individual behavior generally follows the characteristics of conditional cooperation and emotional conditional cooperation. However, it is hard to adequately explain the behavior patterns of conditional cooperation with the evolutionary game theory. This paper introduces expectation-based reinforcement learning methods in the public goods game to investigate and account for the behavior patterns. Instead of letting individuals occupy the entire network as previous studies have done, we focus on studying individual behavior patterns on a network with empty nodes. The results under total population density show the effectivity of our model as they are consistent with those of the previous studies, that is, individuals' behavior exhibits conditional cooperation and its variant moody conditional cooperation. However, in the network with empty nodes, conditional cooperation shows opposite trends. We finally demonstrate that an appropriate population density can facilitate the maintenance and development of cooperation. [ABSTRACT FROM AUTHOR]