Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual
  Transfer of Vision-Language Models   This paper studies zero-shot cross-lingual transfer of vision-language
models. Specifically, we focus on multilingual text-to-video search and propose
a Transformer-based model that learns contextualized multilingual multimodal
embeddings. Under a zero-shot setting, we empirically demonstrate that
performance degrades significantly when we query the multilingual text-video
model with non-English sentences. To address this problem, we introduce a
multilingual multimodal pre-training strategy, and collect a new multilingual
instructional video dataset (MultiHowTo100M) for pre-training. Experiments on
VTT show that our method significantly improves video search in non-English
languages without additional annotations. Furthermore, when multilingual
annotations are available, our method outperforms recent baselines by a large
margin in multilingual text-to-video search on VTT and VATEX; as well as in
multilingual text-to-image search on Multi30K. Our model and Multi-HowTo100M is
available at http://github.com/berniebear/Multi-HT100M.

30
The categories of this paper are: q-alg
23
The categories of this paper are: solv-int
11
The categories of this paper are: hep-lat
19
The categories of this paper are: eess
24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph
37
The categories of this paper are: supr-con
7
The categories of this paper are: hep-th
2
The categories of this paper are: math
12
The categories of this paper are: nucl-th
33
The categories of this paper are: plasm-ph
25
The categories of this paper are: patt-sol
17
The categories of this paper are: q-fin
31
The categories of this paper are: ao-sci
21
The categories of this paper are: adap-org
10
The categories of this paper are: q-bio