Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Shapley Counterfactual Credits for Multi-Agent Reinforcement Learning   Centralized Training with Decentralized Execution (CTDE) has been a popular
paradigm in cooperative Multi-Agent Reinforcement Learning (MARL) settings and
is widely used in many real applications. One of the major challenges in the
training process is credit assignment, which aims to deduce the contributions
of each agent according to the global rewards. Existing credit assignment
methods focus on either decomposing the joint value function into individual
value functions or measuring the impact of local observations and actions on
the global value function. These approaches lack a thorough consideration of
the complicated interactions among multiple agents, leading to an unsuitable
assignment of credit and subsequently mediocre results on MARL. We propose
Shapley Counterfactual Credit Assignment, a novel method for explicit credit
assignment which accounts for the coalition of agents. Specifically, Shapley
Value and its desired properties are leveraged in deep MARL to credit any
combinations of agents, which grants us the capability to estimate the
individual credit for each agent. Despite this capability, the main technical
difficulty lies in the computational complexity of Shapley Value who grows
factorially as the number of agents. We instead utilize an approximation method
via Monte Carlo sampling, which reduces the sample complexity while maintaining
its effectiveness. We evaluate our method on StarCraft II benchmarks across
different scenarios. Our method outperforms existing cooperative MARL
algorithms significantly and achieves the state-of-the-art, with especially
large margins on tasks with more severe difficulties.

35
The categories of this paper are: mtrl-th
15
The categories of this paper are: nucl-ex
26
The categories of this paper are: bayes-an
37
The categories of this paper are: supr-con
19
The categories of this paper are: eess
8
The categories of this paper are: hep-ex
5
The categories of this paper are: gr-qc
24
The categories of this paper are: dg-ga
27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
20
The categories of this paper are: acc-phys
11
The categories of this paper are: hep-lat
21
The categories of this paper are: adap-org
7
The categories of this paper are: hep-th
16
The categories of this paper are: stat
31
The categories of this paper are: ao-sci