Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Scene-Intuitive Agent for Remote Embodied Visual Grounding   Humans learn from life events to form intuitions towards the understanding of
visual environments and languages. Envision that you are instructed by a
high-level instruction, "Go to the bathroom in the master bedroom and replace
the blue towel on the left wall", what would you possibly do to carry out the
task? Intuitively, we comprehend the semantics of the instruction to form an
overview of where a bathroom is and what a blue towel is in mind; then, we
navigate to the target location by consistently matching the bathroom
appearance in mind with the current scene. In this paper, we present an agent
that mimics such human behaviors. Specifically, we focus on the Remote Embodied
Visual Referring Expression in Real Indoor Environments task, called REVERIE,
where an agent is asked to correctly localize a remote target object specified
by a concise high-level natural language instruction, and propose a two-stage
training pipeline. In the first stage, we pretrain the agent with two
cross-modal alignment sub-tasks, namely the Scene Grounding task and the Object
Grounding task. The agent learns where to stop in the Scene Grounding task and
what to attend to in the Object Grounding task respectively. Then, to generate
action sequences, we propose a memory-augmented attentive action decoder to
smoothly fuse the pre-trained vision and language representations with the
agent's past memory experiences. Without bells and whistles, experimental
results show that our method outperforms previous state-of-the-art(SOTA)
significantly, demonstrating the effectiveness of our method.

10
The categories of this paper are: q-bio
7
The categories of this paper are: hep-th
17
The categories of this paper are: q-fin
23
The categories of this paper are: solv-int
8
The categories of this paper are: hep-ex
29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
37
The categories of this paper are: supr-con
9
The categories of this paper are: nlin
14
The categories of this paper are: math-ph
0
The categories of this paper are: hep-ph
16
The categories of this paper are: stat
28
The categories of this paper are: alg-geom
12
The categories of this paper are: nucl-th
26
The categories of this paper are: bayes-an
32
The categories of this paper are: atom-ph