Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: End-to-End Egospheric Spatial Memory   Spatial memory, or the ability to remember and recall specific locations and
objects, is central to autonomous agents' ability to carry out tasks in real
environments. However, most existing artificial memory modules are not very
adept at storing spatial information. We propose a parameter-free module,
Egospheric Spatial Memory (ESM), which encodes the memory in an ego-sphere
around the agent, enabling expressive 3D representations. ESM can be trained
end-to-end via either imitation or reinforcement learning, and improves both
training efficiency and final performance against other memory baselines on
both drone and manipulator visuomotor control tasks. The explicit egocentric
geometry also enables us to seamlessly combine the learned controller with
other non-learned modalities, such as local obstacle avoidance. We further show
applications to semantic segmentation on the ScanNet dataset, where ESM
naturally combines image-level and map-level inference modalities. Through our
broad set of experiments, we show that ESM provides a general computation graph
for embodied spatial reasoning, and the module forms a bridge between real-time
mapping systems and differentiable memory architectures. Implementation at:
https://github.com/ivy-dl/memory.

31
The categories of this paper are: ao-sci
19
The categories of this paper are: eess
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th
20
The categories of this paper are: acc-phys
18
The categories of this paper are: econ
13
The categories of this paper are: quant-ph
34
The categories of this paper are: chem-ph
14
The categories of this paper are: math-ph
10
The categories of this paper are: q-bio
15
The categories of this paper are: nucl-ex
9
The categories of this paper are: nlin
5
The categories of this paper are: gr-qc
25
The categories of this paper are: patt-sol
37
The categories of this paper are: supr-con
8
The categories of this paper are: hep-ex