Towards Distributed Control Under Deficient Communication with Multi-agent Reinforcement Learning

Multi-agent reinforcement learning solves optimization problems in sequential decision-making and enables controlling spatially distributed actuators. We consider a cooperative setting where agents can exchange information via a central controller, e.g. a cloud-based service. In real world applications however, communication channels are often error-prone and agents may become disconnected and can neither send its observation nor receive observations from other agents. We formalize this problem as a subclass of decentralized Markov decision processes and discuss the complexity of the problem. We then propose several solution concepts that involve breaking down the complexity by considering only a subset of failure scenarios, learning independent policies for each failure scenario, reconstructing missing information and learning policies that incorporate the state uncertainty in the training process.