Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Investigating Catastrophic Forgetting During Continual Training for
  Neural Machine Translation   Neural machine translation (NMT) models usually suffer from catastrophic
forgetting during continual training where the models tend to gradually forget
previously learned knowledge and swing to fit the newly added data which may
have a different distribution, e.g. a different domain. Although many methods
have been proposed to solve this problem, we cannot get to know what causes
this phenomenon yet. Under the background of domain adaptation, we investigate
the cause of catastrophic forgetting from the perspectives of modules and
parameters (neurons). The investigation on the modules of the NMT model shows
that some modules have tight relation with the general-domain knowledge while
some other modules are more essential in the domain adaptation. And the
investigation on the parameters shows that some parameters are important for
both the general-domain and in-domain translation and the great change of them
during continual training brings about the performance decline in
general-domain. We conduct experiments across different language pairs and
domains to ensure the validity and reliability of our findings.

19
The categories of this paper are: eess
27
The categories of this paper are: comp-gas
31
The categories of this paper are: ao-sci
34
The categories of this paper are: chem-ph
37
The categories of this paper are: supr-con
8
The categories of this paper are: hep-ex
32
The categories of this paper are: atom-ph
12
The categories of this paper are: nucl-th
11
The categories of this paper are: hep-lat
10
The categories of this paper are: q-bio
2
The categories of this paper are: math
21
The categories of this paper are: adap-org
22
The categories of this paper are: chao-dyn
9
The categories of this paper are: nlin
24
The categories of this paper are: dg-ga
6
The categories of this paper are: astro-ph