Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Unifying Behavioral and Response Diversity for Open-ended Learning in
  Zero-sum Games   Measuring and promoting policy diversity is critical for solving games with
strong non-transitive dynamics where strategic cycles exist, and there is no
consistent winner (e.g., Rock-Paper-Scissors). With that in mind, maintaining a
pool of diverse policies via open-ended learning is an attractive solution,
which can generate auto-curricula to avoid being exploited. However, in
conventional open-ended learning algorithms, there are no widely accepted
definitions for diversity, making it hard to construct and evaluate the diverse
policies. In this work, we summarize previous concepts of diversity and work
towards offering a unified measure of diversity in multi-agent open-ended
learning to include all elements in Markov games, based on both Behavioral
Diversity (BD) and Response Diversity (RD). At the trajectory distribution
level, we re-define BD in the state-action space as the discrepancies of
occupancy measures. For the reward dynamics, we propose RD to characterize
diversity through the responses of policies when encountering different
opponents. We also show that many current diversity measures fall in one of the
categories of BD or RD but not both. With this unified diversity measure, we
design the corresponding diversity-promoting objective and population
effectivity when seeking the best responses in open-ended learning. We validate
our methods in both relatively simple games like matrix game, non-transitive
mixture model, and the complex \textit{Google Research Football} environment.
The population found by our methods reveals the lowest exploitability, highest
population effectivity in matrix game and non-transitive mixture model, as well
as the largest goal difference when interacting with opponents of various
levels in \textit{Google Research Football}.

31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
30
The categories of this paper are: q-alg
25
The categories of this paper are: patt-sol
24
The categories of this paper are: dg-ga
21
The categories of this paper are: adap-org
13
The categories of this paper are: quant-ph
2
The categories of this paper are: math
14
The categories of this paper are: math-ph
16
The categories of this paper are: stat
29
The categories of this paper are: funct-an
35
The categories of this paper are: mtrl-th
6
The categories of this paper are: astro-ph