Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Unsupervised Bitext Mining and Translation via Self-trained Contextual
  Embeddings   We describe an unsupervised method to create pseudo-parallel corpora for
machine translation (MT) from unaligned text. We use multilingual BERT to
create source and target sentence embeddings for nearest-neighbor search and
adapt the model via self-training. We validate our technique by extracting
parallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a
24.5 point increase (absolute) in F1 scores over previous unsupervised methods.
We then improve an XLM-based unsupervised neural MT system pre-trained on
Wikipedia by supplementing it with pseudo-parallel text mined from the same
corpus, boosting unsupervised translation performance by up to 3.5 BLEU on the
WMT'14 French-English and WMT'16 German-English tasks and outperforming the
previous state-of-the-art. Finally, we enrich the IWSLT'15 English-Vietnamese
corpus with pseudo-parallel Wikipedia sentence pairs, yielding a 1.2 BLEU
improvement on the low-resource MT task. We demonstrate that unsupervised
bitext mining is an effective way of augmenting MT datasets and complements
existing techniques like initializing with pre-trained contextual embeddings.

9
The categories of this paper are: nlin
25
The categories of this paper are: patt-sol
30
The categories of this paper are: q-alg
2
The categories of this paper are: math
26
The categories of this paper are: bayes-an
12
The categories of this paper are: nucl-th
19
The categories of this paper are: eess
8
The categories of this paper are: hep-ex
27
The categories of this paper are: comp-gas
16
The categories of this paper are: stat
6
The categories of this paper are: astro-ph
11
The categories of this paper are: hep-lat
35
The categories of this paper are: mtrl-th
31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
34
The categories of this paper are: chem-ph