Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Deep Multimodal Neural Architecture Search   Designing effective neural networks is fundamentally important in deep
multimodal learning. Most existing works focus on a single task and design
neural architectures manually, which are highly task-specific and hard to
generalize to different tasks. In this paper, we devise a generalized deep
multimodal neural architecture search (MMnas) framework for various multimodal
learning tasks. Given multimodal input, we first define a set of primitive
operations, and then construct a deep encoder-decoder based unified backbone,
where each encoder or decoder block corresponds to an operation searched from a
predefined operation pool. On top of the unified backbone, we attach
task-specific heads to tackle different multimodal learning tasks. By using a
gradient-based NAS algorithm, the optimal architectures for different tasks are
learned efficiently. Extensive ablation studies, comprehensive analysis, and
comparative experimental results show that the obtained MMnasNet significantly
outperforms existing state-of-the-art approaches across three multimodal
learning tasks (over five datasets), including visual question answering,
image-text matching, and visual grounding.

9
The categories of this paper are: nlin
24
The categories of this paper are: dg-ga
32
The categories of this paper are: atom-ph
0
The categories of this paper are: hep-ph
12
The categories of this paper are: nucl-th
4
The categories of this paper are: cond-mat
28
The categories of this paper are: alg-geom
20
The categories of this paper are: acc-phys
2
The categories of this paper are: math
29
The categories of this paper are: funct-an
17
The categories of this paper are: q-fin
18
The categories of this paper are: econ
30
The categories of this paper are: q-alg
8
The categories of this paper are: hep-ex
26
The categories of this paper are: bayes-an
11
The categories of this paper are: hep-lat