  In multi-agent reinforcement learning (MARL), independent learning (IL) often
shows remarkable performance and easily scales with the number of agents. Yet,
using IL can be inefficient and runs the risk of failing to successfully train,
particularly in scenarios that require agents to coordinate their actions.
Using centralised learning (CL) enables MARL agents to quickly learn how to
coordinate their behaviour but employing CL everywhere is often prohibitively
expensive in real-world applications. Besides, using CL in value-based methods
often needs strong representational constraints (e.g. individual-global-max
condition) that can lead to poor performance if violated. In this paper, we
introduce a novel plug & play IL framework named Multi-Agent Network Selection
Algorithm (MANSA) which selectively employs CL only at states that require
coordination. At its core, MANSA has an additional agent that uses switching
controls to quickly learn the best states to activate CL during training, using
CL only where necessary and vastly reducing the computational burden of CL. Our
theory proves MANSA preserves cooperative MARL convergence properties, boosts
IL performance and can optimally make use of a fixed budget on the number CL
calls. We show empirically in Level-based Foraging (LBF) and StarCraft
Multi-agent Challenge (SMAC) that MANSA achieves fast, superior and more
reliable performance while making 40% fewer CL calls in SMAC and using CL at
only 1% CL calls in LBF.
