Controlling cooperative behaviors in real-time environments using reinforcement learning

Abstract

The research presented in this Master’s Thesis expands on previous research in the Artificial Intelligence (AI) domain of cooperative, deep, multi-agent reinforcement learning (MARL) in a common-pool resource environment. Specifically, we expand on the Forest Fire Model (FFM) presented in Cooperative Control of Environmental Extremes by Artificial Intelligent Agents. Previous research contributes an improved understanding of emergent cooperative behaviors for AI agents through the lens of game-theoretic concepts of equilibria (in the context of ecological engineering). This work presented in this thesis focuses on increasing realism in the FFM simulation by modeling the probability of ignition of fires inspired by real-world data. While the original research kept fire ignition as probabilistically constant, here we explore how periods of stress and abundance (i.e. seasonal fire impacts) guide agent behaviors. Notably, this research shows that the duration of organizational behavior and the degree to which it occurs can be controlled through manipulation of probabilistic occurrence of negative rewards - explored here as dynamic probabilistic fire functions. Understanding and controlling cooperative behaviors provides relevant solutions for addressing resource sharing among self-interested parties and serves as foundational work for climate change oriented solutions.