Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: UNIMO: Towards Unified-Modal Understanding and Generation via
  Cross-Modal Contrastive Learning   Existed pre-training methods either focus on single-modal tasks or
multi-modal tasks, and cannot effectively adapt to each other. They can only
utilize single-modal data (i.e. text or image) or limited multi-modal data
(i.e. image-text pairs). In this work, we propose a unified-modal pre-training
architecture, namely UNIMO, which can effectively adapt to both single-modal
and multi-modal understanding and generation tasks. Large scale of free text
corpus and image collections can be utilized to improve the capability of
visual and textual understanding, and cross-modal contrastive learning (CMCL)
is leveraged to align the textual and visual information into a unified
semantic space over a corpus of image-text pairs. As the non-paired
single-modal data is very rich, our model can utilize much larger scale of data
to learn more generalizable representations. Moreover, the textual knowledge
and visual knowledge can enhance each other in the unified semantic space. The
experimental results show that UNIMO significantly improves the performance of
several single-modal and multi-modal downstream tasks. Our code and pre-trained
models are public at the UNIMO project page https://unimo-ptm.github.io/

33
The categories of this paper are: plasm-ph
8
The categories of this paper are: hep-ex
2
The categories of this paper are: math
13
The categories of this paper are: quant-ph
35
The categories of this paper are: mtrl-th
22
The categories of this paper are: chao-dyn
29
The categories of this paper are: funct-an
5
The categories of this paper are: gr-qc
19
The categories of this paper are: eess
0
The categories of this paper are: hep-ph
3
The categories of this paper are: physics
11
The categories of this paper are: hep-lat
17
The categories of this paper are: q-fin
28
The categories of this paper are: alg-geom
14
The categories of this paper are: math-ph
6
The categories of this paper are: astro-ph