Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games   Proximal Policy Optimization (PPO) is a popular on-policy reinforcement
learning algorithm but is significantly less utilized than off-policy learning
algorithms in multi-agent settings. This is often due the belief that on-policy
methods are significantly less sample efficient than their off-policy
counterparts in multi-agent problems. In this work, we investigate Multi-Agent
PPO (MAPPO), a variant of PPO which is specialized for multi-agent settings.
Using a 1-GPU desktop, we show that MAPPO achieves surprisingly strong
performance in three popular multi-agent testbeds: the particle-world
environments, the Starcraft multi-agent challenge, and the Hanabi challenge,
with minimal hyperparameter tuning and without any domain-specific algorithmic
modifications or architectures. In the majority of environments, we find that
compared to off-policy baselines, MAPPO achieves strong results while
exhibiting comparable sample efficiency. Finally, through ablation studies, we
present the implementation and algorithmic factors which are most influential
to MAPPO's practical performance.

14
The categories of this paper are: math-ph
9
The categories of this paper are: nlin
33
The categories of this paper are: plasm-ph
23
The categories of this paper are: solv-int
4
The categories of this paper are: cond-mat
19
The categories of this paper are: eess
10
The categories of this paper are: q-bio
27
The categories of this paper are: comp-gas
25
The categories of this paper are: patt-sol
34
The categories of this paper are: chem-ph
0
The categories of this paper are: hep-ph
37
The categories of this paper are: supr-con
28
The categories of this paper are: alg-geom
22
The categories of this paper are: chao-dyn
24
The categories of this paper are: dg-ga
13
The categories of this paper are: quant-ph