  This paper investigates the multi-agent cooperative exploration problem,
which requires multiple agents to explore an unseen environment via sensory
signals in a limited time. A popular approach to exploration tasks is to
combine active mapping with planning. Metric maps capture the details of the
spatial representation, but are with high communication traffic and may vary
significantly between scenarios, resulting in inferior generalization.
Topological maps are a promising alternative as they consist only of nodes and
edges with abstract but essential information and are less influenced by the
scene structures. However, most existing topology-based exploration tasks
utilize classical methods for planning, which are time-consuming and
sub-optimal due to their handcrafted design. Deep reinforcement learning (DRL)
has shown great potential for learning (near) optimal policies through fast
end-to-end inference. In this paper, we propose Multi-Agent Neural Topological
Mapping (MANTM) to improve exploration efficiency and generalization for
multi-agent exploration tasks. MANTM mainly comprises a Topological Mapper and
a novel RL-based Hierarchical Topological Planner (HTP). The Topological Mapper
employs a visual encoder and distance-based heuristics to construct a graph
containing main nodes and their corresponding ghost nodes. The HTP leverages
graph neural networks to capture correlations between agents and graph nodes in
a coarse-to-fine manner for effective global goal selection. Extensive
experiments conducted in a physically-realistic simulator, Habitat, demonstrate
that MANTM reduces the steps by at least 26.40% over planning-based baselines
and by at least 7.63% over RL-based competitors in unseen scenarios.
