Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: How to Sense the World: Leveraging Hierarchy in Multimodal Perception
  for Robust Reinforcement Learning Agents   This work addresses the problem of sensing the world: how to learn a
multimodal representation of a reinforcement learning agent's environment that
allows the execution of tasks under incomplete perceptual conditions. To
address such problem, we argue for hierarchy in the design of representation
models and contribute with a novel multimodal representation model, MUSE. The
proposed model learns hierarchical representations: low-level modality-specific
representations, encoded from raw observation data, and a high-level multimodal
representation, encoding joint-modality information to allow robust state
estimation. We employ MUSE as the sensory representation model of deep
reinforcement learning agents provided with multimodal observations in Atari
games. We perform a comparative study over different designs of reinforcement
learning agents, showing that MUSE allows agents to perform tasks under
incomplete perceptual experience with minimal performance loss. Finally, we
evaluate the performance of MUSE in literature-standard multimodal scenarios
with higher number and more complex modalities, showing that it outperforms
state-of-the-art multimodal variational autoencoders in single and
cross-modality generation.

4
The categories of this paper are: cond-mat
9
The categories of this paper are: nlin
28
The categories of this paper are: alg-geom
3
The categories of this paper are: physics
10
The categories of this paper are: q-bio
13
The categories of this paper are: quant-ph
0
The categories of this paper are: hep-ph
23
The categories of this paper are: solv-int
17
The categories of this paper are: q-fin
27
The categories of this paper are: comp-gas
24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph
33
The categories of this paper are: plasm-ph
2
The categories of this paper are: math
15
The categories of this paper are: nucl-ex
37
The categories of this paper are: supr-con