Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multi-Agent Constrained Policy Optimisation   Developing reinforcement learning algorithms that satisfy safety constraints
is becoming increasingly important in real-world applications. In multi-agent
reinforcement learning (MARL) settings, policy optimisation with safety
awareness is particularly challenging because each individual agent has to not
only meet its own safety constraints, but also consider those of others so that
their joint behaviour can be guaranteed safe. Despite its importance, the
problem of safe multi-agent learning has not been rigorously studied; very few
solutions have been proposed, nor a sharable testing environment or benchmarks.
To fill these gaps, in this work, we formulate the safe MARL problem as a
constrained Markov game and solve it with policy optimisation methods. Our
solutions -- Multi-Agent Constrained Policy Optimisation (MACPO) and
MAPPO-Lagrangian -- leverage the theories from both constrained policy
optimisation and multi-agent trust region learning. Crucially, our methods
enjoy theoretical guarantees of both monotonic improvement in reward and
satisfaction of safety constraints at every iteration. To examine the
effectiveness of our methods, we develop the benchmark suite of Safe
Multi-Agent MuJoCo that involves a variety of MARL baselines. Experimental
results justify that MACPO/MAPPO-Lagrangian can consistently satisfy safety
constraints, meanwhile achieving comparable performance to strong baselines.

23
The categories of this paper are: solv-int
3
The categories of this paper are: physics
35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas
22
The categories of this paper are: chao-dyn
7
The categories of this paper are: hep-th
28
The categories of this paper are: alg-geom
19
The categories of this paper are: eess
4
The categories of this paper are: cond-mat
25
The categories of this paper are: patt-sol
14
The categories of this paper are: math-ph
30
The categories of this paper are: q-alg
37
The categories of this paper are: supr-con
21
The categories of this paper are: adap-org
12
The categories of this paper are: nucl-th
20
The categories of this paper are: acc-phys