Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence
  Pretraining   Despite the success of multilingual sequence-to-sequence pretraining, most
existing approaches rely on monolingual corpora, and do not make use of the
strong cross-lingual signal contained in parallel data. In this paper, we
present PARADISE (PARAllel & Denoising Integration in SEquence-to-sequence
models), which extends the conventional denoising objective used to train these
models by (i) replacing words in the noised sequence according to a
multilingual dictionary, and (ii) predicting the reference translation
according to a parallel corpus instead of recovering the original sequence. Our
experiments on machine translation and cross-lingual natural language inference
show an average improvement of 2.0 BLEU points and 6.7 accuracy points from
integrating parallel data into pretraining, respectively, obtaining results
that are competitive with several popular models at a fraction of their
computational cost.

32
The categories of this paper are: atom-ph
25
The categories of this paper are: patt-sol
22
The categories of this paper are: chao-dyn
0
The categories of this paper are: hep-ph
8
The categories of this paper are: hep-ex
4
The categories of this paper are: cond-mat
15
The categories of this paper are: nucl-ex
5
The categories of this paper are: gr-qc
7
The categories of this paper are: hep-th
12
The categories of this paper are: nucl-th
17
The categories of this paper are: q-fin
28
The categories of this paper are: alg-geom
35
The categories of this paper are: mtrl-th
29
The categories of this paper are: funct-an
13
The categories of this paper are: quant-ph
11
The categories of this paper are: hep-lat