Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Learning Nearly Decomposable Value Functions Via Communication
  Minimization   Reinforcement learning encounters major challenges in multi-agent settings,
such as scalability and non-stationarity. Recently, value function
factorization learning emerges as a promising way to address these challenges
in collaborative multi-agent systems. However, existing methods have been
focusing on learning fully decentralized value functions, which are not
efficient for tasks requiring communication. To address this limitation, this
paper presents a novel framework for learning nearly decomposable Q-functions
(NDQ) via communication minimization, with which agents act on their own most
of the time but occasionally send messages to other agents in order for
effective coordination. This framework hybridizes value function factorization
learning and communication learning by introducing two information-theoretic
regularizers. These regularizers are maximizing mutual information between
agents' action selection and communication messages while minimizing the
entropy of messages between agents. We show how to optimize these regularizers
in a way that is easily integrated with existing value function factorization
methods such as QMIX. Finally, we demonstrate that, on the StarCraft unit
micromanagement benchmark, our framework significantly outperforms baseline
methods and allows us to cut off more than $80\%$ of communication without
sacrificing the performance. The videos of our experiments are available at
https://sites.google.com/view/ndq.

35
The categories of this paper are: mtrl-th
24
The categories of this paper are: dg-ga
13
The categories of this paper are: quant-ph
17
The categories of this paper are: q-fin
21
The categories of this paper are: adap-org
23
The categories of this paper are: solv-int
10
The categories of this paper are: q-bio
29
The categories of this paper are: funct-an
9
The categories of this paper are: nlin
28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th
7
The categories of this paper are: hep-th
25
The categories of this paper are: patt-sol
0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
4
The categories of this paper are: cond-mat