Cooperation is the rule, not the exception: reinforcement learning in the Battle of the Sexes

Abstract

Society is highly influenced by conventions, which are a form of cooperation. In many situations, individuals act together for the benefit of the group. This phenomenon is easy to understand when all individuals share the same interest. However, when there exists conflict, it is not clear if altruism is required or pure self-interest can lead to cooperation. The repeated version of the Battle of the Sexes game can summarize this situation. Although conflict is present, players need to cooperate to obtain good rewards. Here we show experimentally that two selfish reinforcement learning agents learn to cooperate in this conflictive scenario. We found that two Q-learning agents playing this game modeled as a Markov Game reach a cooperative fair solution. That is, two agents that learn based solely on their own self-interest end up cooperating. Furthermore, we found that Q-learning is able to converge in this multi-agent situation. Our results demonstrate that cooperation among individuals in this particular conflictive scenario can be explained by means of pure self-interest. Moreover, cooperation in this setting is the rule, not the exception as the convergence to it is robust to parameter asymmetry between agents. We also introduced opponent modeling into the players as a Beta binomial model. It worked well in modeling the adversary but agents fail to properly exploit that knowledge.