Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Image Difference Captioning with Pre-training and Contrastive Learning   The Image Difference Captioning (IDC) task aims to describe the visual
differences between two similar images with natural language. The major
challenges of this task lie in two aspects: 1) fine-grained visual differences
that require learning stronger vision and language association and 2) high-cost
of manual annotations that leads to limited supervised data. To address these
challenges, we propose a new modeling framework following the
pre-training-finetuning paradigm. Specifically, we design three self-supervised
tasks and contrastive learning strategies to align visual differences and text
descriptions at a fine-grained level. Moreover, we propose a data expansion
strategy to utilize extra cross-task supervision information, such as data for
fine-grained image classification, to alleviate the limitation of available
supervised IDC data. Extensive experiments on two IDC benchmark datasets,
CLEVR-Change and Birds-to-Words, demonstrate the effectiveness of the proposed
modeling framework. The codes and models will be released at
https://github.com/yaolinli/IDC.

29
The categories of this paper are: funct-an
30
The categories of this paper are: q-alg
21
The categories of this paper are: adap-org
17
The categories of this paper are: q-fin
2
The categories of this paper are: math
25
The categories of this paper are: patt-sol
37
The categories of this paper are: supr-con
5
The categories of this paper are: gr-qc
33
The categories of this paper are: plasm-ph
15
The categories of this paper are: nucl-ex
14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
24
The categories of this paper are: dg-ga
27
The categories of this paper are: comp-gas
7
The categories of this paper are: hep-th
4
The categories of this paper are: cond-mat