  Evolutionary science provides evidence that diversity confers resilience.
Yet, traditional multi-agent reinforcement learning techniques commonly enforce
homogeneity to increase training sample efficiency. When a system of learning
agents is not constrained to homogeneous policies, individual agents may
develop diverse behaviors, resulting in emergent complementarity that benefits
the system. Despite this feat, there is a surprising lack of tools that measure
behavioral diversity in systems of learning agents. Such techniques would pave
the way towards understanding the impact of diversity in collective resilience
and performance. In this paper, we introduce System Neural Diversity (SND): a
measure of behavioral heterogeneity for multi-agent systems where agents have
stochastic policies. %over a continuous state space. We discuss and prove its
theoretical properties, and compare it with alternate, state-of-the-art
behavioral diversity metrics used in cross-disciplinary domains. Through
simulations of a variety of multi-agent tasks, we show how our metric
constitutes an important diagnostic tool to analyze latent properties of
behavioral heterogeneity. By comparing SND with task reward in static tasks,
where the problem does not change during training, we show that it is key to
understanding the effectiveness of heterogeneous vs homogeneous agents. In
dynamic tasks, where the problem is affected by repeated disturbances during
training, we show that heterogeneous agents are first able to learn specialized
roles that allow them to cope with the disturbance, and then retain these roles
when the disturbance is removed. SND allows a direct measurement of this latent
resilience, while other proxies such as task performance (reward) fail to.
