Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Do all Roads Lead to Rome? Understanding the Role of Initialization in
  Iterative Back-Translation   Back-translation provides a simple yet effective approach to exploit
monolingual corpora in Neural Machine Translation (NMT). Its iterative variant,
where two opposite NMT models are jointly trained by alternately using a
synthetic parallel corpus generated by the reverse model, plays a central role
in unsupervised machine translation. In order to start producing sound
translations and provide a meaningful training signal to each other, existing
approaches rely on either a separate machine translation system to warm up the
iterative procedure, or some form of pre-training to initialize the weights of
the model. In this paper, we analyze the role that such initialization plays in
iterative back-translation. Is the behavior of the final system heavily
dependent on it? Or does iterative back-translation converge to a similar
solution given any reasonable initialization? Through a series of empirical
experiments over a diverse set of warmup systems, we show that, although the
quality of the initial system does affect final performance, its effect is
relatively small, as iterative back-translation has a strong tendency to
convergence to a similar solution. As such, the margin of improvement left for
the initialization method is narrow, suggesting that future research should
focus more on improving the iterative mechanism itself.

11
The categories of this paper are: hep-lat
37
The categories of this paper are: supr-con
24
The categories of this paper are: dg-ga
21
The categories of this paper are: adap-org
12
The categories of this paper are: nucl-th
6
The categories of this paper are: astro-ph
5
The categories of this paper are: gr-qc
22
The categories of this paper are: chao-dyn
34
The categories of this paper are: chem-ph
20
The categories of this paper are: acc-phys
27
The categories of this paper are: comp-gas
26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
25
The categories of this paper are: patt-sol
35
The categories of this paper are: mtrl-th
4
The categories of this paper are: cond-mat