text: Many tasks in AI require the collaboration of multiple agents. Typically, the communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines. In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.
  @cite_13: Realization of cooperative behavior in multi-agent system is important for improving problem solving ability. Reinforcement learning is one of the learning methods for such cooperative behavior of agents. In this paper, we consider pursuit problem for multi-agent reinforcement learning with communication between the agents. In our study, the agents obtain communication codes through learning. Here, the codes are rules for communicating appropriate information under various situations. We call the learning of communication codes signal learning. The signal is expressed by bit sequence, and its length is set to be variable. We carried out experiment for performance comparison with varying the signal length from 0 to 4 bits. As a result, it has been shown that, in learning precision, the case of 1 bit or more bits communication outperformed the case of no communication. It also has been shown that 4 bits communication produced the best result among the five cases, while learning with longer signals required much more iterations.
  @cite_25: We analyze a general model of multi-agent communication in which all agents communicate simultaneously to a message board. A genetic algorithm is used to learn multi-agent languages for the predator agents in a version of the predator-prey problem. The resulting evolved behavior of the communicating multi-agent system is equivalent to that of a Mealy machine whose states are determined by the evolved language. We also constructed non-learning predators whose capture behavior was designed to take advantage of prey behavior known a priori. Simulations show that introducing noise to the decision process of the hard-coded predators allow them to significantly ourperform all previously published work on similar preys. Furthermore, the evolved communicating predators were able to perform significantly better than the hard-coded predators, which indicates that the system was able to learn superior communicating strategies not readily available to the human designer.
  @cite_12: We present a principled and efficient planning algorithm for cooperative multiagent dynamic systems. A striking feature of our method is that the coordination and communication between the agents is not imposed, but derived directly from the system dynamics and function approximation architecture. We view the entire multiagent system as a single, large Markov decision process (MDP), which we assume can be represented in a factored way using a dynamic Bayesian network (DBN). The action space of the resulting MDP is the joint action space of the entire set of agents. Our approach is based on the use of factored linear value functions as an approximation to the joint value function. This factorization of the value function allows the agents to coordinate their actions at runtime using a natural message passing scheme. We provide a simple and efficient method for computing such an approximate value function by solving a single linear program, whose size is determined by the interaction between the value function structure and the DBN. We thereby avoid the exponential blowup in the state and action space. We show that our approach compares favorably with approaches based on reward sharing. We also show that our algorithm is an efficient alternative to more complicated algorithms even in the single agent case.
  @cite_2: 
summary: A few notable approaches involve learning to communicate between agents under partial visibility: Kasai al @cite_13 and Varshavskaya al @cite_2 , both use distributed tabular-RL approaches for simulated tasks. Giles & Jim @cite_25 use an evolutionary algorithm, rather than reinforcement learning. Guestrin al @cite_12 use a single large MDP to control a collection of agents, via a factored message passing framework where the messages are learned. In contrast to these approaches, our model uses a deep network for both agent control and communication.