  Multi-agent systems often require agents to collaborate with or compete
against other agents with diverse goals, behaviors, or strategies. Agent
modeling is essential when designing adaptive policies for intelligent machine
agents in multiagent systems, as this is the means by which the ego agent
understands other agents' behavior and extracts their meaningful policy
representations. These representations can be used to enhance the ego agent's
adaptive policy which is trained by reinforcement learning. However, existing
agent modeling approaches typically assume the availability of local
observations from other agents (modeled agents) during training or a long
observation trajectory for policy adaption. To remove these constrictive
assumptions and improve agent modeling performance, we devised a Contrastive
Learning-based Agent Modeling (CLAM) method that relies only on the local
observations from the ego agent during training and execution. With these
observations, CLAM is capable of generating consistent high-quality policy
representations in real-time right from the beginning of each episode. We
evaluated the efficacy of our approach in both cooperative and competitive
multi-agent environments. Our experiments demonstrate that our approach
achieves state-of-the-art on both cooperative and competitive tasks,
highlighting the potential of contrastive learning-based agent modeling for
enhancing reinforcement learning.
