  The ability of modeling the other agents, such as understanding their
intentions and skills, is essential to an agent's interactions with other
agents. Conventional agent modeling relies on passive observation from
demonstrations. In this work, we propose an interactive agent modeling scheme
enabled by encouraging an agent to learn to probe. In particular, the probing
agent (i.e. a learner) learns to interact with the environment and with a
target agent (i.e., a demonstrator) to maximize the change in the observed
behaviors of that agent. Through probing, rich behaviors can be observed and
are used for enhancing the agent modeling to learn a more accurate mind model
of the target agent. Our framework consists of two learning processes: i)
imitation learning for an approximated agent model and ii) pure
curiosity-driven reinforcement learning for an efficient probing policy to
discover new behaviors that otherwise can not be observed. We have validated
our approach in four different tasks. The experimental results suggest that the
agent model learned by our approach i) generalizes better in novel scenarios
than the ones learned by passive observation, random probing, and other
curiosity-driven approaches do, and ii) can be used for enhancing performance
in multiple applications including distilling optimal planning to a policy net,
collaboration, and competition. A video demo is available at
https://www.dropbox.com/s/8mz6rd3349tso67/Probing_Demo.mov?dl=0
