  In many applications involving multi-agent system (MAS), it is imperative to
test an experimental (Exp) autonomous agent in a high-fidelity simulator prior
to its deployment to production, to avoid unexpected losses in the real-world.
Such a simulator acts as the environmental background (BG) agent(s), called
agent-based simulator (ABS), aiming to replicate the complex real MAS. However,
developing realistic ABS remains challenging, mainly due to the sequential and
dynamic nature of such systems. To fill this gap, we propose a metric to
distinguish between real and synthetic multi-agent systems, which is evaluated
through the live interaction between the Exp and BG agents to explicitly
account for the systems' sequential nature. Specifically, we characterize the
system/environment by studying the effect of a sequence of BG agents' responses
to the environment state evolution and take such effects' differences as MAS
distance metric; The effect estimation is cast as a causal inference problem
since the environment evolution is confounded with the previous environment
state. Importantly, we propose the Interactive Agent-Guided Simulation (INTAGS)
framework to build a realistic ABS by optimizing over this novel metric. To
adapt to any environment with interactive sequential decision making agents,
INTAGS formulates the simulator as a stochastic policy in reinforcement
learning. Moreover, INTAGS utilizes the policy gradient update to bypass
differentiating the proposed metric such that it can support non-differentiable
operations of multi-agent environments. Through extensive experiments, we
demonstrate the effectiveness of INTAGS on an equity stock market simulation
example. We show that using INTAGS to calibrate the simulator can generate more
realistic market data compared to the state-of-the-art conditional Wasserstein
Generative Adversarial Network approach.
