  Mean Field Control (MFC) is a powerful approximation tool to solve
large-scale Multi-Agent Reinforcement Learning (MARL) problems. However, the
success of MFC relies on the presumption that given the local states and
actions of all the agents, the next (local) states of the agents evolve
conditionally independent of each other. Here we demonstrate that even in a
MARL setting where agents share a common global state in addition to their
local states evolving conditionally independently (thus introducing a
correlation between the state transition processes of individual agents), the
MFC can still be applied as a good approximation tool. The global state is
assumed to be non-decomposable i.e., it cannot be expressed as a collection of
local states of the agents. We compute the approximation error as
$\mathcal{O}(e)$ where $e=\frac{1}{\sqrt{N}}\left[\sqrt{|\mathcal{X}|}
+\sqrt{|\mathcal{U}|}\right]$. The size of the agent population is denoted by
the term $N$, and $|\mathcal{X}|, |\mathcal{U}|$ respectively indicate the
sizes of (local) state and action spaces of individual agents. The
approximation error is found to be independent of the size of the shared global
state space. We further demonstrate that in a special case if the reward and
state transition functions are independent of the action distribution of the
population, then the error can be improved to
$e=\frac{\sqrt{|\mathcal{X}|}}{\sqrt{N}}$. Finally, we devise a Natural Policy
Gradient based algorithm that solves the MFC problem with
$\mathcal{O}(\epsilon^{-3})$ sample complexity and obtains a policy that is
within $\mathcal{O}(\max\{e,\epsilon\})$ error of the optimal MARL policy for
any $\epsilon>0$.
