Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multimodal Generative Models for Compositional Representation Learning   As deep neural networks become more adept at traditional tasks, many of the
most exciting new challenges concern multimodality---observations that combine
diverse types, such as image and text. In this paper, we introduce a family of
multimodal deep generative models derived from variational bounds on the
evidence (data marginal likelihood). As part of our derivation we find that
many previous multimodal variational autoencoders used objectives that do not
correctly bound the joint marginal likelihood across modalities. We further
generalize our objective to work with several types of deep generative model
(VAE, GAN, and flow-based), and allow use of different model types for
different modalities. We benchmark our models across many image, label, and
text datasets, and find that our multimodal VAEs excel with and without weak
supervision. Additional improvements come from use of GAN image models with VAE
language models. Finally, we investigate the effect of language on learned
image representations through a variety of downstream tasks, such as
compositionally, bounding box prediction, and visual relation prediction. We
find evidence that these image representations are more abstract and
compositional than equivalent representations learned from only visual data.

30
The categories of this paper are: q-alg
15
The categories of this paper are: nucl-ex
24
The categories of this paper are: dg-ga
31
The categories of this paper are: ao-sci
28
The categories of this paper are: alg-geom
25
The categories of this paper are: patt-sol
2
The categories of this paper are: math
17
The categories of this paper are: q-fin
9
The categories of this paper are: nlin
32
The categories of this paper are: atom-ph
8
The categories of this paper are: hep-ex
21
The categories of this paper are: adap-org
20
The categories of this paper are: acc-phys
22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc
7
The categories of this paper are: hep-th