Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Bilingual Dictionary-based Language Model Pretraining for Neural Machine
  Translation   Recent studies have demonstrated a perceivable improvement on the performance
of neural machine translation by applying cross-lingual language model
pretraining (Lample and Conneau, 2019), especially the Translation Language
Modeling (TLM). To alleviate the need for expensive parallel corpora by TLM, in
this work, we incorporate the translation information from dictionaries into
the pretraining process and propose a novel Bilingual Dictionary-based Language
Model (BDLM). We evaluate our BDLM in Chinese, English, and Romanian. For
Chinese-English, we obtained a 55.0 BLEU on WMT-News19 (Tiedemann, 2012) and a
24.3 BLEU on WMT20 news-commentary, outperforming the Vanilla Transformer
(Vaswani et al., 2017) by more than 8.4 BLEU and 2.3 BLEU, respectively.
According to our results, the BDLM also has advantages on convergence speed and
predicting rare words. The increase in BLEU for WMT16 Romanian-English also
shows its effectiveness in low-resources language translation.

35
The categories of this paper are: mtrl-th
30
The categories of this paper are: q-alg
5
The categories of this paper are: gr-qc
33
The categories of this paper are: plasm-ph
23
The categories of this paper are: solv-int
36
The categories of this paper are: cmp-lg
31
The categories of this paper are: ao-sci
11
The categories of this paper are: hep-lat
22
The categories of this paper are: chao-dyn
0
The categories of this paper are: hep-ph
4
The categories of this paper are: cond-mat
27
The categories of this paper are: comp-gas
10
The categories of this paper are: q-bio
25
The categories of this paper are: patt-sol
14
The categories of this paper are: math-ph
16
The categories of this paper are: stat