Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: CUPID: Adaptive Curation of Pre-training Data for Video-and-Language
  Representation Learning   This work concerns video-language pre-training and representation learning.
In this now ubiquitous training scheme, a model first performs pre-training on
paired videos and text (e.g., video clips and accompanied subtitles) from a
large uncurated source corpus, before transferring to specific downstream
tasks. This two-stage training process inevitably raises questions about the
generalization ability of the pre-trained model, which is particularly
pronounced when a salient domain gap exists between source and target data
(e.g., instructional cooking videos vs. movies). In this paper, we first bring
to light the sensitivity of pre-training objectives (contrastive vs.
reconstructive) to domain discrepancy. Then, we propose a simple yet effective
framework, CUPID, to bridge this domain gap by filtering and adapting source
data to the target data, followed by domain-focused pre-training. Comprehensive
experiments demonstrate that pre-training on a considerably small subset of
domain-focused data can effectively close the source-target domain gap and
achieve significant performance gain, compared to random sampling or even
exploiting the full pre-training dataset. CUPID yields new state-of-the-art
performance across multiple video-language and video tasks, including
text-to-video retrieval [72, 37], video question answering [36], and video
captioning [72], with consistent performance lift over different pre-training
methods.

13
The categories of this paper are: quant-ph
16
The categories of this paper are: stat
28
The categories of this paper are: alg-geom
18
The categories of this paper are: econ
34
The categories of this paper are: chem-ph
32
The categories of this paper are: atom-ph
3
The categories of this paper are: physics
27
The categories of this paper are: comp-gas
6
The categories of this paper are: astro-ph
35
The categories of this paper are: mtrl-th
24
The categories of this paper are: dg-ga
25
The categories of this paper are: patt-sol
26
The categories of this paper are: bayes-an
2
The categories of this paper are: math
33
The categories of this paper are: plasm-ph
21
The categories of this paper are: adap-org